{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this walkthrough, we will develop a neural Sentiment Analysis model for Norwegian without access to labelled data. The first thing we will do is to read in our data and get it into Spacy DocBins:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "import spacy\n",
    "from spacy.tokens import DocBin\n",
    "import pandas as pd\n",
    "try:\n",
    "    import transformers\n",
    "except ImportError as e:\n",
    "    print(e, ', which is required to run this notebook')\n",
    "\n",
    "nlp = spacy.load(\"nb_core_news_md\")\n",
    "\n",
    "train_doc_bin = DocBin(store_user_data=True)\n",
    "dev_doc_bin = DocBin(store_user_data=True)\n",
    "test_doc_bin = DocBin(store_user_data=True)\n",
    "\n",
    "train = pd.read_csv(\"../../data/sentiment/norec_sentence/train.txt\", delimiter=\"\\t\", header=None)\n",
    "dev = pd.read_csv(\"../../data/sentiment/norec_sentence/dev.txt\", delimiter=\"\\t\", header=None)\n",
    "test = pd.read_csv(\"../../data/sentiment/norec_sentence/test.txt\", delimiter=\"\\t\", header=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We load the data as a `pandas.DataFrame`, where the first column is the sentence index, the second is the polarity (0=Negative, 1=Neutral, 2=Positive), and the third is the pretokenized sentence, as a string. We then create a Spacy DocBin object for train, dev, and test splits."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   0                                                  1\n",
      "0  1                                Stor og bred Seagal\n",
      "1  1  Steven Seagal er blitt like stor som Travolta ...\n",
      "2  1  I denne filmen handler det om en seriemorder m...\n",
      "3  1   Dermed er genrens patologi-imperativ ivaretatt .\n",
      "4  0                     Verre er det med slagsmålene .\n",
      "5  0  Klipperen har overtatt Seagals martial art , o...\n",
      "6  1                              Men noen fikse ting :\n",
      "7  1  Hettemannen ser ut som David Beckham etter en ...\n",
      "8  1                  Seagal får en Dirty Harry-pakke :\n",
      "9  2  Ei finslig dame skal bli med på kjøret og obse...\n"
     ]
    }
   ],
   "source": [
    "print(train.head(10))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "for sid, (label, sent) in train.iterrows():\n",
    "    doc = nlp(sent)\n",
    "    doc.user_data[\"gold\"] = label\n",
    "    train_doc_bin.add(doc)\n",
    "train_doc_bin.to_disk(\"../../data/sentiment/norec_sentence/train.docbin\")\n",
    "\n",
    "for sid, (label, sent) in dev.iterrows():\n",
    "    doc = nlp(sent)\n",
    "    doc.user_data[\"gold\"] = label\n",
    "    dev_doc_bin.add(doc)\n",
    "dev_doc_bin.to_disk(\"../../data/sentiment/norec_sentence/dev.docbin\")\n",
    "\n",
    "for sid, (label, sent) in test.iterrows():\n",
    "    doc = nlp(sent)\n",
    "    doc.user_data[\"gold\"] = label\n",
    "    test_doc_bin.add(doc)\n",
    "test_doc_bin.to_disk(\"../../data/sentiment/norec_sentence/test.docbin\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will begin to build up our labelling functions. We will start by training a document-level Bag-Of-Words (BOW) model on example Norwegian Agency for Exchange Cooperation (NoReC) documents. These are given star ratings from 1-6, so the labelling function will have to map these to negative, neutral, and positive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting model on ../../data/sentiment/conllu.tar.gz\n",
      "Doc-level F1: 0.313\n",
      "Saving vectorizer and model to ../../data/sentiment/models/doc\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "sys.path.insert(0, '../..')\n",
    "import skweak\n",
    "from sklearn.metrics import f1_score\n",
    "from sentiment_models import DocBOWAnnotator\n",
    "\n",
    "dann = DocBOWAnnotator(\"doc_norec\",\n",
    "                       model_path=\"../../data/sentiment/models/doc\",\n",
    "                       doclevel_data=\"../../data/sentiment/conllu.tar.gz\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will train a multi-lingual BERT model on the Stanford Sentiment Treebank."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of the model checkpoint at bert-base-multilingual-uncased were not used when initializing BertForSequenceClassification: ['cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias']\n",
      "- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
      "Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-multilingual-uncased and are newly initialized: ['classifier.weight', 'classifier.bias']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n",
      "  0%|          | 0/534 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training for 20 epochs...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 16%|█▋        | 88/534 [05:41<28:53,  3.89s/it]\n"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-5-e8a11d64f799>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     10\u001b[0m \u001b[0mtokenizer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mBertTokenizer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfrom_pretrained\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"bert-base-multilingual-uncased\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     11\u001b[0m \u001b[0mmodel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mBertForSequenceClassification\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfrom_pretrained\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"bert-base-multilingual-uncased\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_labels\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 12\u001b[0;31m \u001b[0mtrain\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtokenizer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtrain_loader\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdev_loader\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"../../data/sentiment/models/mbert-sst\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/Exps/skweak/domains/sentiment/transformer_model.py\u001b[0m in \u001b[0;36mtrain\u001b[0;34m(model, tokenizer, train_loader, dev_loader, save_dir)\u001b[0m\n\u001b[1;32m    120\u001b[0m             \u001b[0mencoding\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtokenizer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msents\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mreturn_tensors\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'pt'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpadding\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtruncation\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmax_length\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m150\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    121\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 122\u001b[0;31m             \u001b[0moutputs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0mencoding\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    123\u001b[0m             \u001b[0m_\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0moutputs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlogits\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    124\u001b[0m             \u001b[0mtrain_preds\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mextend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtolist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Exps/skweak/domains/myvenv/lib/python3.9/site-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m    887\u001b[0m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_slow_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    888\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 889\u001b[0;31m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    890\u001b[0m         for hook in itertools.chain(\n\u001b[1;32m    891\u001b[0m                 \u001b[0m_global_forward_hooks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Exps/skweak/domains/myvenv/lib/python3.9/site-packages/transformers/models/bert/modeling_bert.py\u001b[0m in \u001b[0;36mforward\u001b[0;34m(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, labels, output_attentions, output_hidden_states, return_dict)\u001b[0m\n\u001b[1;32m   1499\u001b[0m         \u001b[0mreturn_dict\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mreturn_dict\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mreturn_dict\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconfig\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0muse_return_dict\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1500\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1501\u001b[0;31m         outputs = self.bert(\n\u001b[0m\u001b[1;32m   1502\u001b[0m             \u001b[0minput_ids\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1503\u001b[0m             \u001b[0mattention_mask\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mattention_mask\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Exps/skweak/domains/myvenv/lib/python3.9/site-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m    887\u001b[0m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_slow_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    888\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 889\u001b[0;31m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    890\u001b[0m         for hook in itertools.chain(\n\u001b[1;32m    891\u001b[0m                 \u001b[0m_global_forward_hooks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Exps/skweak/domains/myvenv/lib/python3.9/site-packages/transformers/models/bert/modeling_bert.py\u001b[0m in \u001b[0;36mforward\u001b[0;34m(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask, past_key_values, use_cache, output_attentions, output_hidden_states, return_dict)\u001b[0m\n\u001b[1;32m    969\u001b[0m             \u001b[0mpast_key_values_length\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpast_key_values_length\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    970\u001b[0m         )\n\u001b[0;32m--> 971\u001b[0;31m         encoder_outputs = self.encoder(\n\u001b[0m\u001b[1;32m    972\u001b[0m             \u001b[0membedding_output\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    973\u001b[0m             \u001b[0mattention_mask\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mextended_attention_mask\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Exps/skweak/domains/myvenv/lib/python3.9/site-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m    887\u001b[0m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_slow_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    888\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 889\u001b[0;31m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    890\u001b[0m         for hook in itertools.chain(\n\u001b[1;32m    891\u001b[0m                 \u001b[0m_global_forward_hooks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Exps/skweak/domains/myvenv/lib/python3.9/site-packages/transformers/models/bert/modeling_bert.py\u001b[0m in \u001b[0;36mforward\u001b[0;34m(self, hidden_states, attention_mask, head_mask, encoder_hidden_states, encoder_attention_mask, past_key_values, use_cache, output_attentions, output_hidden_states, return_dict)\u001b[0m\n\u001b[1;32m    566\u001b[0m                 )\n\u001b[1;32m    567\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 568\u001b[0;31m                 layer_outputs = layer_module(\n\u001b[0m\u001b[1;32m    569\u001b[0m                     \u001b[0mhidden_states\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    570\u001b[0m                     \u001b[0mattention_mask\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Exps/skweak/domains/myvenv/lib/python3.9/site-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m    887\u001b[0m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_slow_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    888\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 889\u001b[0;31m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    890\u001b[0m         for hook in itertools.chain(\n\u001b[1;32m    891\u001b[0m                 \u001b[0m_global_forward_hooks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Exps/skweak/domains/myvenv/lib/python3.9/site-packages/transformers/models/bert/modeling_bert.py\u001b[0m in \u001b[0;36mforward\u001b[0;34m(self, hidden_states, attention_mask, head_mask, encoder_hidden_states, encoder_attention_mask, past_key_value, output_attentions)\u001b[0m\n\u001b[1;32m    454\u001b[0m         \u001b[0;31m# decoder uni-directional self-attention cached key/values tuple is at positions 1,2\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    455\u001b[0m         \u001b[0mself_attn_past_key_value\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpast_key_value\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mpast_key_value\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 456\u001b[0;31m         self_attention_outputs = self.attention(\n\u001b[0m\u001b[1;32m    457\u001b[0m             \u001b[0mhidden_states\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    458\u001b[0m             \u001b[0mattention_mask\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Exps/skweak/domains/myvenv/lib/python3.9/site-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m    887\u001b[0m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_slow_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    888\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 889\u001b[0;31m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    890\u001b[0m         for hook in itertools.chain(\n\u001b[1;32m    891\u001b[0m                 \u001b[0m_global_forward_hooks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Exps/skweak/domains/myvenv/lib/python3.9/site-packages/transformers/models/bert/modeling_bert.py\u001b[0m in \u001b[0;36mforward\u001b[0;34m(self, hidden_states, attention_mask, head_mask, encoder_hidden_states, encoder_attention_mask, past_key_value, output_attentions)\u001b[0m\n\u001b[1;32m    385\u001b[0m         \u001b[0moutput_attentions\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    386\u001b[0m     ):\n\u001b[0;32m--> 387\u001b[0;31m         self_outputs = self.self(\n\u001b[0m\u001b[1;32m    388\u001b[0m             \u001b[0mhidden_states\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    389\u001b[0m             \u001b[0mattention_mask\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Exps/skweak/domains/myvenv/lib/python3.9/site-packages/torch/nn/modules/module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[0;34m(self, *input, **kwargs)\u001b[0m\n\u001b[1;32m    887\u001b[0m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_slow_forward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    888\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 889\u001b[0;31m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    890\u001b[0m         for hook in itertools.chain(\n\u001b[1;32m    891\u001b[0m                 \u001b[0m_global_forward_hooks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Exps/skweak/domains/myvenv/lib/python3.9/site-packages/transformers/models/bert/modeling_bert.py\u001b[0m in \u001b[0;36mforward\u001b[0;34m(self, hidden_states, attention_mask, head_mask, encoder_hidden_states, encoder_attention_mask, past_key_value, output_attentions)\u001b[0m\n\u001b[1;32m    323\u001b[0m             \u001b[0mattention_probs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mattention_probs\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mhead_mask\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    324\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 325\u001b[0;31m         \u001b[0mcontext_layer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmatmul\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mattention_probs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalue_layer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    326\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    327\u001b[0m         \u001b[0mcontext_layer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcontext_layer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpermute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m3\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcontiguous\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "from transformer_model import SSTDataLoader, train\n",
    "from transformers import BertTokenizer, BertForSequenceClassification\n",
    "\n",
    "train_loader = SSTDataLoader(\"../../data/sentiment/sst/train.txt\")\n",
    "dev_loader = SSTDataLoader(\"../../data/sentiment/sst/dev.txt\")\n",
    "test_loader = SSTDataLoader(\"../../data/sentiment/sst/test.txt\")\n",
    "\n",
    "tokenizer = BertTokenizer.from_pretrained(\"bert-base-multilingual-uncased\")\n",
    "model = BertForSequenceClassification.from_pretrained(\"bert-base-multilingual-uncased\", num_labels=3)\n",
    "train(model, tokenizer, train_loader, dev_loader, \"../../data/sentiment/models/mbert-sst\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can construct a full annotator with all annotators described above, and then run it on the NoReC sentence-level dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading lexicon functions\n",
      "Loading learned sentiment model functions\n",
      "Loaded model from ../../data/sentiment/models/doc\n",
      "Loaded nlptown/bert-base-multilingual-uncased-sentiment\n",
      "Loaded mBERT from ../../data/sentiment/models/mbert-sst\n",
      "Number of processed documents: 1\n",
      "Number of processed documents: 2\n",
      "Number of processed documents: 3\n",
      "Number of processed documents: 4\n",
      "Number of processed documents: 5\n",
      "Number of processed documents: 6\n",
      "Number of processed documents: 7\n",
      "Number of processed documents: 8\n",
      "Number of processed documents: 9\n",
      "Number of processed documents: 10\n",
      "Number of processed documents: 11\n",
      "Number of processed documents: 12\n",
      "Number of processed documents: 13\n",
      "Number of processed documents: 14\n",
      "Number of processed documents: 15\n",
      "Number of processed documents: 16\n",
      "Number of processed documents: 17\n",
      "Number of processed documents: 18\n",
      "Number of processed documents: 19\n",
      "Number of processed documents: 20\n",
      "Number of processed documents: 21\n",
      "Number of processed documents: 22\n",
      "Number of processed documents: 23\n",
      "Number of processed documents: 24\n",
      "Number of processed documents: 25\n",
      "Number of processed documents: 26\n",
      "Number of processed documents: 27\n",
      "Number of processed documents: 28\n",
      "Number of processed documents: 29\n",
      "Number of processed documents: 30\n",
      "Number of processed documents: 31\n",
      "Number of processed documents: 32\n",
      "Number of processed documents: 33\n",
      "Number of processed documents: 34\n",
      "Number of processed documents: 35\n",
      "Number of processed documents: 36\n",
      "Number of processed documents: 37\n",
      "Number of processed documents: 38\n",
      "Number of processed documents: 39\n",
      "Number of processed documents: 40\n",
      "Number of processed documents: 41\n",
      "Number of processed documents: 42\n",
      "Number of processed documents: 43\n",
      "Number of processed documents: 44\n",
      "Number of processed documents: 45\n",
      "Number of processed documents: 46\n",
      "Number of processed documents: 47\n",
      "Number of processed documents: 48\n",
      "Number of processed documents: 49\n",
      "Number of processed documents: 50\n",
      "Number of processed documents: 51\n",
      "Number of processed documents: 52\n",
      "Number of processed documents: 53\n",
      "Number of processed documents: 54\n",
      "Number of processed documents: 55\n",
      "Number of processed documents: 56\n",
      "Number of processed documents: 57\n",
      "Number of processed documents: 58\n",
      "Number of processed documents: 59\n",
      "Number of processed documents: 60\n",
      "Number of processed documents: 61\n",
      "Number of processed documents: 62\n",
      "Number of processed documents: 63\n",
      "Number of processed documents: 64\n",
      "Number of processed documents: 65\n",
      "Number of processed documents: 66\n",
      "Number of processed documents: 67\n",
      "Number of processed documents: 68\n",
      "Number of processed documents: 69\n",
      "Number of processed documents: 70\n",
      "Number of processed documents: 71\n",
      "Number of processed documents: 72\n",
      "Number of processed documents: 73\n",
      "Number of processed documents: 74\n",
      "Number of processed documents: 75\n",
      "Number of processed documents: 76\n",
      "Number of processed documents: 77\n",
      "Number of processed documents: 78\n",
      "Number of processed documents: 79\n",
      "Number of processed documents: 80\n",
      "Number of processed documents: 81\n",
      "Number of processed documents: 82\n",
      "Number of processed documents: 83\n",
      "Number of processed documents: 84\n",
      "Number of processed documents: 85\n",
      "Number of processed documents: 86\n",
      "Number of processed documents: 87\n",
      "Number of processed documents: 88\n",
      "Number of processed documents: 89\n",
      "Number of processed documents: 90\n",
      "Number of processed documents: 91\n",
      "Number of processed documents: 92\n",
      "Number of processed documents: 93\n",
      "Number of processed documents: 94\n",
      "Number of processed documents: 95\n",
      "Number of processed documents: 96\n",
      "Number of processed documents: 97\n",
      "Number of processed documents: 98\n",
      "Number of processed documents: 99\n",
      "Number of processed documents: 100\n",
      "Number of processed documents: 101\n",
      "Number of processed documents: 102\n",
      "Number of processed documents: 103\n",
      "Number of processed documents: 104\n",
      "Number of processed documents: 105\n",
      "Number of processed documents: 106\n",
      "Number of processed documents: 107\n",
      "Number of processed documents: 108\n",
      "Number of processed documents: 109\n",
      "Number of processed documents: 110\n",
      "Number of processed documents: 111\n",
      "Number of processed documents: 112\n",
      "Number of processed documents: 113\n",
      "Number of processed documents: 114\n",
      "Number of processed documents: 115\n",
      "Number of processed documents: 116\n",
      "Number of processed documents: 117\n",
      "Number of processed documents: 118\n",
      "Number of processed documents: 119\n",
      "Number of processed documents: 120\n",
      "Number of processed documents: 121\n",
      "Number of processed documents: 122\n",
      "Number of processed documents: 123\n",
      "Number of processed documents: 124\n",
      "Number of processed documents: 125\n",
      "Number of processed documents: 126\n",
      "Number of processed documents: 127\n",
      "Number of processed documents: 128\n",
      "Number of processed documents: 129\n",
      "Number of processed documents: 130\n",
      "Number of processed documents: 131\n",
      "Number of processed documents: 132\n",
      "Number of processed documents: 133\n",
      "Number of processed documents: 134\n",
      "Number of processed documents: 135\n",
      "Number of processed documents: 136\n",
      "Number of processed documents: 137\n",
      "Number of processed documents: 138\n",
      "Number of processed documents: 139\n",
      "Number of processed documents: 140\n",
      "Number of processed documents: 141\n",
      "Number of processed documents: 142\n",
      "Number of processed documents: 143\n",
      "Number of processed documents: 144\n",
      "Number of processed documents: 145\n",
      "Number of processed documents: 146\n",
      "Number of processed documents: 147\n",
      "Number of processed documents: 148\n",
      "Number of processed documents: 149\n",
      "Number of processed documents: 150\n",
      "Number of processed documents: 151\n",
      "Number of processed documents: 152\n",
      "Number of processed documents: 153\n",
      "Number of processed documents: 154\n",
      "Number of processed documents: 155\n",
      "Number of processed documents: 156\n",
      "Number of processed documents: 157\n",
      "Number of processed documents: 158\n",
      "Number of processed documents: 159\n",
      "Number of processed documents: 160\n",
      "Number of processed documents: 161\n",
      "Number of processed documents: 162\n",
      "Number of processed documents: 163\n",
      "Number of processed documents: 164\n",
      "Number of processed documents: 165\n",
      "Number of processed documents: 166\n",
      "Number of processed documents: 167\n",
      "Number of processed documents: 168\n",
      "Number of processed documents: 169\n",
      "Number of processed documents: 170\n",
      "Number of processed documents: 171\n",
      "Number of processed documents: 172\n",
      "Number of processed documents: 173\n",
      "Number of processed documents: 174\n",
      "Number of processed documents: 175\n",
      "Number of processed documents: 176\n",
      "Number of processed documents: 177\n",
      "Number of processed documents: 178\n",
      "Number of processed documents: 179\n",
      "Number of processed documents: 180\n",
      "Number of processed documents: 181\n",
      "Number of processed documents: 182\n",
      "Number of processed documents: 183\n",
      "Number of processed documents: 184\n",
      "Number of processed documents: 185\n",
      "Number of processed documents: 186\n",
      "Number of processed documents: 187\n",
      "Number of processed documents: 188\n",
      "Number of processed documents: 189\n",
      "Number of processed documents: 190\n",
      "Number of processed documents: 191\n",
      "Number of processed documents: 192\n",
      "Number of processed documents: 193\n",
      "Number of processed documents: 194\n",
      "Number of processed documents: 195\n",
      "Number of processed documents: 196\n",
      "Number of processed documents: 197\n",
      "Number of processed documents: 198\n",
      "Number of processed documents: 199\n",
      "Number of processed documents: 200\n",
      "Number of processed documents: 201\n",
      "Number of processed documents: 202\n",
      "Number of processed documents: 203\n",
      "Number of processed documents: 204\n",
      "Number of processed documents: 205\n",
      "Number of processed documents: 206\n",
      "Number of processed documents: 207\n",
      "Number of processed documents: 208\n",
      "Number of processed documents: 209\n",
      "Number of processed documents: 210\n",
      "Number of processed documents: 211\n",
      "Number of processed documents: 212\n",
      "Number of processed documents: 213\n",
      "Number of processed documents: 214\n",
      "Number of processed documents: 215\n",
      "Number of processed documents: 216\n",
      "Number of processed documents: 217\n",
      "Number of processed documents: 218\n",
      "Number of processed documents: 219\n",
      "Number of processed documents: 220\n",
      "Number of processed documents: 221\n",
      "Number of processed documents: 222\n",
      "Number of processed documents: 223\n",
      "Number of processed documents: 224\n",
      "Number of processed documents: 225\n",
      "Number of processed documents: 226\n",
      "Number of processed documents: 227\n",
      "Number of processed documents: 228\n",
      "Number of processed documents: 229\n",
      "Number of processed documents: 230\n",
      "Number of processed documents: 231\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 232\n",
      "Number of processed documents: 233\n",
      "Number of processed documents: 234\n",
      "Number of processed documents: 235\n",
      "Number of processed documents: 236\n",
      "Number of processed documents: 237\n",
      "Number of processed documents: 238\n",
      "Number of processed documents: 239\n",
      "Number of processed documents: 240\n",
      "Number of processed documents: 241\n",
      "Number of processed documents: 242\n",
      "Number of processed documents: 243\n",
      "Number of processed documents: 244\n",
      "Number of processed documents: 245\n",
      "Number of processed documents: 246\n",
      "Number of processed documents: 247\n",
      "Number of processed documents: 248\n",
      "Number of processed documents: 249\n",
      "Number of processed documents: 250\n",
      "Number of processed documents: 251\n",
      "Number of processed documents: 252\n",
      "Number of processed documents: 253\n",
      "Number of processed documents: 254\n",
      "Number of processed documents: 255\n",
      "Number of processed documents: 256\n",
      "Number of processed documents: 257\n",
      "Number of processed documents: 258\n",
      "Number of processed documents: 259\n",
      "Number of processed documents: 260\n",
      "Number of processed documents: 261\n",
      "Number of processed documents: 262\n",
      "Number of processed documents: 263\n",
      "Number of processed documents: 264\n",
      "Number of processed documents: 265\n",
      "Number of processed documents: 266\n",
      "Number of processed documents: 267\n",
      "Number of processed documents: 268\n",
      "Number of processed documents: 269\n",
      "Number of processed documents: 270\n",
      "Number of processed documents: 271\n",
      "Number of processed documents: 272\n",
      "Number of processed documents: 273\n",
      "Number of processed documents: 274\n",
      "Number of processed documents: 275\n",
      "Number of processed documents: 276\n",
      "Number of processed documents: 277\n",
      "Number of processed documents: 278\n",
      "Number of processed documents: 279\n",
      "Number of processed documents: 280\n",
      "Number of processed documents: 281\n",
      "Number of processed documents: 282\n",
      "Number of processed documents: 283\n",
      "Number of processed documents: 284\n",
      "Number of processed documents: 285\n",
      "Number of processed documents: 286\n",
      "Number of processed documents: 287\n",
      "Number of processed documents: 288\n",
      "Number of processed documents: 289\n",
      "Number of processed documents: 290\n",
      "Number of processed documents: 291\n",
      "Number of processed documents: 292\n",
      "Number of processed documents: 293\n",
      "Number of processed documents: 294\n",
      "Number of processed documents: 295\n",
      "Number of processed documents: 296\n",
      "Number of processed documents: 297\n",
      "Number of processed documents: 298\n",
      "Number of processed documents: 299\n",
      "Number of processed documents: 300\n",
      "Number of processed documents: 301\n",
      "Number of processed documents: 302\n",
      "Number of processed documents: 303\n",
      "Number of processed documents: 304\n",
      "Number of processed documents: 305\n",
      "Number of processed documents: 306\n",
      "Number of processed documents: 307\n",
      "Number of processed documents: 308\n",
      "Number of processed documents: 309\n",
      "Number of processed documents: 310\n",
      "Number of processed documents: 311\n",
      "Number of processed documents: 312\n",
      "Number of processed documents: 313\n",
      "Number of processed documents: 314\n",
      "Number of processed documents: 315\n",
      "Number of processed documents: 316\n",
      "Number of processed documents: 317\n",
      "Number of processed documents: 318\n",
      "Number of processed documents: 319\n",
      "Number of processed documents: 320\n",
      "Number of processed documents: 321\n",
      "Number of processed documents: 322\n",
      "Number of processed documents: 323\n",
      "Number of processed documents: 324\n",
      "Number of processed documents: 325\n",
      "Number of processed documents: 326\n",
      "Number of processed documents: 327\n",
      "Number of processed documents: 328\n",
      "Number of processed documents: 329\n",
      "Number of processed documents: 330\n",
      "Number of processed documents: 331\n",
      "Number of processed documents: 332\n",
      "Number of processed documents: 333\n",
      "Number of processed documents: 334\n",
      "Number of processed documents: 335\n",
      "Number of processed documents: 336\n",
      "Number of processed documents: 337\n",
      "Number of processed documents: 338\n",
      "Number of processed documents: 339\n",
      "Number of processed documents: 340\n",
      "Number of processed documents: 341\n",
      "Number of processed documents: 342\n",
      "Number of processed documents: 343\n",
      "Number of processed documents: 344\n",
      "Number of processed documents: 345\n",
      "Number of processed documents: 346\n",
      "Number of processed documents: 347\n",
      "Number of processed documents: 348\n",
      "Number of processed documents: 349\n",
      "Number of processed documents: 350\n",
      "Number of processed documents: 351\n",
      "Number of processed documents: 352\n",
      "Number of processed documents: 353\n",
      "Number of processed documents: 354\n",
      "Number of processed documents: 355\n",
      "Number of processed documents: 356\n",
      "Number of processed documents: 357\n",
      "Number of processed documents: 358\n",
      "Number of processed documents: 359\n",
      "Number of processed documents: 360\n",
      "Number of processed documents: 361\n",
      "Number of processed documents: 362\n",
      "Number of processed documents: 363\n",
      "Number of processed documents: 364\n",
      "Number of processed documents: 365\n",
      "Number of processed documents: 366\n",
      "Number of processed documents: 367\n",
      "Number of processed documents: 368\n",
      "Number of processed documents: 369\n",
      "Number of processed documents: 370\n",
      "Number of processed documents: 371\n",
      "Number of processed documents: 372\n",
      "Number of processed documents: 373\n",
      "Number of processed documents: 374\n",
      "Number of processed documents: 375\n",
      "Number of processed documents: 376\n",
      "Number of processed documents: 377\n",
      "Number of processed documents: 378\n",
      "Number of processed documents: 379\n",
      "Number of processed documents: 380\n",
      "Number of processed documents: 381\n",
      "Number of processed documents: 382\n",
      "Number of processed documents: 383\n",
      "Number of processed documents: 384\n",
      "Number of processed documents: 385\n",
      "Number of processed documents: 386\n",
      "Number of processed documents: 387\n",
      "Number of processed documents: 388\n",
      "Number of processed documents: 389\n",
      "Number of processed documents: 390\n",
      "Number of processed documents: 391\n",
      "Number of processed documents: 392\n",
      "Number of processed documents: 393\n",
      "Number of processed documents: 394\n",
      "Number of processed documents: 395\n",
      "Number of processed documents: 396\n",
      "Number of processed documents: 397\n",
      "Number of processed documents: 398\n",
      "Number of processed documents: 399\n",
      "Number of processed documents: 400\n",
      "Number of processed documents: 401\n",
      "Number of processed documents: 402\n",
      "Number of processed documents: 403\n",
      "Number of processed documents: 404\n",
      "Number of processed documents: 405\n",
      "Number of processed documents: 406\n",
      "Number of processed documents: 407\n",
      "Number of processed documents: 408\n",
      "Number of processed documents: 409\n",
      "Number of processed documents: 410\n",
      "Number of processed documents: 411\n",
      "Number of processed documents: 412\n",
      "Number of processed documents: 413\n",
      "Number of processed documents: 414\n",
      "Number of processed documents: 415\n",
      "Number of processed documents: 416\n",
      "Number of processed documents: 417\n",
      "Number of processed documents: 418\n",
      "Number of processed documents: 419\n",
      "Number of processed documents: 420\n",
      "Number of processed documents: 421\n",
      "Number of processed documents: 422\n",
      "Number of processed documents: 423\n",
      "Number of processed documents: 424\n",
      "Number of processed documents: 425\n",
      "Number of processed documents: 426\n",
      "Number of processed documents: 427\n",
      "Number of processed documents: 428\n",
      "Number of processed documents: 429\n",
      "Number of processed documents: 430\n",
      "Number of processed documents: 431\n",
      "Number of processed documents: 432\n",
      "Number of processed documents: 433\n",
      "Number of processed documents: 434\n",
      "Number of processed documents: 435\n",
      "Number of processed documents: 436\n",
      "Number of processed documents: 437\n",
      "Number of processed documents: 438\n",
      "Number of processed documents: 439\n",
      "Number of processed documents: 440\n",
      "Number of processed documents: 441\n",
      "Number of processed documents: 442\n",
      "Number of processed documents: 443\n",
      "Number of processed documents: 444\n",
      "Number of processed documents: 445\n",
      "Number of processed documents: 446\n",
      "Number of processed documents: 447\n",
      "Number of processed documents: 448\n",
      "Number of processed documents: 449\n",
      "Number of processed documents: 450\n",
      "Number of processed documents: 451\n",
      "Number of processed documents: 452\n",
      "Number of processed documents: 453\n",
      "Number of processed documents: 454\n",
      "Number of processed documents: 455\n",
      "Number of processed documents: 456\n",
      "Number of processed documents: 457\n",
      "Number of processed documents: 458\n",
      "Number of processed documents: 459\n",
      "Number of processed documents: 460\n",
      "Number of processed documents: 461\n",
      "Number of processed documents: 462\n",
      "Number of processed documents: 463\n",
      "Number of processed documents: 464\n",
      "Number of processed documents: 465\n",
      "Number of processed documents: 466\n",
      "Number of processed documents: 467\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 468\n",
      "Number of processed documents: 469\n",
      "Number of processed documents: 470\n",
      "Number of processed documents: 471\n",
      "Number of processed documents: 472\n",
      "Number of processed documents: 473\n",
      "Number of processed documents: 474\n",
      "Number of processed documents: 475\n",
      "Number of processed documents: 476\n",
      "Number of processed documents: 477\n",
      "Number of processed documents: 478\n",
      "Number of processed documents: 479\n",
      "Number of processed documents: 480\n",
      "Number of processed documents: 481\n",
      "Number of processed documents: 482\n",
      "Number of processed documents: 483\n",
      "Number of processed documents: 484\n",
      "Number of processed documents: 485\n",
      "Number of processed documents: 486\n",
      "Number of processed documents: 487\n",
      "Number of processed documents: 488\n",
      "Number of processed documents: 489\n",
      "Number of processed documents: 490\n",
      "Number of processed documents: 491\n",
      "Number of processed documents: 492\n",
      "Number of processed documents: 493\n",
      "Number of processed documents: 494\n",
      "Number of processed documents: 495\n",
      "Number of processed documents: 496\n",
      "Number of processed documents: 497\n",
      "Number of processed documents: 498\n",
      "Number of processed documents: 499\n",
      "Number of processed documents: 500\n",
      "Number of processed documents: 501\n",
      "Number of processed documents: 502\n",
      "Number of processed documents: 503\n",
      "Number of processed documents: 504\n",
      "Number of processed documents: 505\n",
      "Number of processed documents: 506\n",
      "Number of processed documents: 507\n",
      "Number of processed documents: 508\n",
      "Number of processed documents: 509\n",
      "Number of processed documents: 510\n",
      "Number of processed documents: 511\n",
      "Number of processed documents: 512\n",
      "Number of processed documents: 513\n",
      "Number of processed documents: 514\n",
      "Number of processed documents: 515\n",
      "Number of processed documents: 516\n",
      "Number of processed documents: 517\n",
      "Number of processed documents: 518\n",
      "Number of processed documents: 519\n",
      "Number of processed documents: 520\n",
      "Number of processed documents: 521\n",
      "Number of processed documents: 522\n",
      "Number of processed documents: 523\n",
      "Number of processed documents: 524\n",
      "Number of processed documents: 525\n",
      "Number of processed documents: 526\n",
      "Number of processed documents: 527\n",
      "Number of processed documents: 528\n",
      "Number of processed documents: 529\n",
      "Number of processed documents: 530\n",
      "Number of processed documents: 531\n",
      "Number of processed documents: 532\n",
      "Number of processed documents: 533\n",
      "Number of processed documents: 534\n",
      "Number of processed documents: 535\n",
      "Number of processed documents: 536\n",
      "Number of processed documents: 537\n",
      "Number of processed documents: 538\n",
      "Number of processed documents: 539\n",
      "Number of processed documents: 540\n",
      "Number of processed documents: 541\n",
      "Number of processed documents: 542\n",
      "Number of processed documents: 543\n",
      "Number of processed documents: 544\n",
      "Number of processed documents: 545\n",
      "Number of processed documents: 546\n",
      "Number of processed documents: 547\n",
      "Number of processed documents: 548\n",
      "Number of processed documents: 549\n",
      "Number of processed documents: 550\n",
      "Number of processed documents: 551\n",
      "Number of processed documents: 552\n",
      "Number of processed documents: 553\n",
      "Number of processed documents: 554\n",
      "Number of processed documents: 555\n",
      "Number of processed documents: 556\n",
      "Number of processed documents: 557\n",
      "Number of processed documents: 558\n",
      "Number of processed documents: 559\n",
      "Number of processed documents: 560\n",
      "Number of processed documents: 561\n",
      "Number of processed documents: 562\n",
      "Number of processed documents: 563\n",
      "Number of processed documents: 564\n",
      "Number of processed documents: 565\n",
      "Number of processed documents: 566\n",
      "Number of processed documents: 567\n",
      "Number of processed documents: 568\n",
      "Number of processed documents: 569\n",
      "Number of processed documents: 570\n",
      "Number of processed documents: 571\n",
      "Number of processed documents: 572\n",
      "Number of processed documents: 573\n",
      "Number of processed documents: 574\n",
      "Number of processed documents: 575\n",
      "Number of processed documents: 576\n",
      "Number of processed documents: 577\n",
      "Number of processed documents: 578\n",
      "Number of processed documents: 579\n",
      "Number of processed documents: 580\n",
      "Number of processed documents: 581\n",
      "Number of processed documents: 582\n",
      "Number of processed documents: 583\n",
      "Number of processed documents: 584\n",
      "Number of processed documents: 585\n",
      "Number of processed documents: 586\n",
      "Number of processed documents: 587\n",
      "Number of processed documents: 588\n",
      "Number of processed documents: 589\n",
      "Number of processed documents: 590\n",
      "Number of processed documents: 591\n",
      "Number of processed documents: 592\n",
      "Number of processed documents: 593\n",
      "Number of processed documents: 594\n",
      "Number of processed documents: 595\n",
      "Number of processed documents: 596\n",
      "Number of processed documents: 597\n",
      "Number of processed documents: 598\n",
      "Number of processed documents: 599\n",
      "Number of processed documents: 600\n",
      "Number of processed documents: 601\n",
      "Number of processed documents: 602\n",
      "Number of processed documents: 603\n",
      "Number of processed documents: 604\n",
      "Number of processed documents: 605\n",
      "Number of processed documents: 606\n",
      "Number of processed documents: 607\n",
      "Number of processed documents: 608\n",
      "Number of processed documents: 609\n",
      "Number of processed documents: 610\n",
      "Number of processed documents: 611\n",
      "Number of processed documents: 612\n",
      "Number of processed documents: 613\n",
      "Number of processed documents: 614\n",
      "Number of processed documents: 615\n",
      "Number of processed documents: 616\n",
      "Number of processed documents: 617\n",
      "Number of processed documents: 618\n",
      "Number of processed documents: 619\n",
      "Number of processed documents: 620\n",
      "Number of processed documents: 621\n",
      "Number of processed documents: 622\n",
      "Number of processed documents: 623\n",
      "Number of processed documents: 624\n",
      "Number of processed documents: 625\n",
      "Number of processed documents: 626\n",
      "Number of processed documents: 627\n",
      "Number of processed documents: 628\n",
      "Number of processed documents: 629\n",
      "Number of processed documents: 630\n",
      "Number of processed documents: 631\n",
      "Number of processed documents: 632\n",
      "Number of processed documents: 633\n",
      "Number of processed documents: 634\n",
      "Number of processed documents: 635\n",
      "Number of processed documents: 636\n",
      "Number of processed documents: 637\n",
      "Number of processed documents: 638\n",
      "Number of processed documents: 639\n",
      "Number of processed documents: 640\n",
      "Number of processed documents: 641\n",
      "Number of processed documents: 642\n",
      "Number of processed documents: 643\n",
      "Number of processed documents: 644\n",
      "Number of processed documents: 645\n",
      "Number of processed documents: 646\n",
      "Number of processed documents: 647\n",
      "Number of processed documents: 648\n",
      "Number of processed documents: 649\n",
      "Number of processed documents: 650\n",
      "Number of processed documents: 651\n",
      "Number of processed documents: 652\n",
      "Number of processed documents: 653\n",
      "Number of processed documents: 654\n",
      "Number of processed documents: 655\n",
      "Number of processed documents: 656\n",
      "Number of processed documents: 657\n",
      "Number of processed documents: 658\n",
      "Number of processed documents: 659\n",
      "Number of processed documents: 660\n",
      "Number of processed documents: 661\n",
      "Number of processed documents: 662\n",
      "Number of processed documents: 663\n",
      "Number of processed documents: 664\n",
      "Number of processed documents: 665\n",
      "Number of processed documents: 666\n",
      "Number of processed documents: 667\n",
      "Number of processed documents: 668\n",
      "Number of processed documents: 669\n",
      "Number of processed documents: 670\n",
      "Number of processed documents: 671\n",
      "Number of processed documents: 672\n",
      "Number of processed documents: 673\n",
      "Number of processed documents: 674\n",
      "Number of processed documents: 675\n",
      "Number of processed documents: 676\n",
      "Number of processed documents: 677\n",
      "Number of processed documents: 678\n",
      "Number of processed documents: 679\n",
      "Number of processed documents: 680\n",
      "Number of processed documents: 681\n",
      "Number of processed documents: 682\n",
      "Number of processed documents: 683\n",
      "Number of processed documents: 684\n",
      "Number of processed documents: 685\n",
      "Number of processed documents: 686\n",
      "Number of processed documents: 687\n",
      "Number of processed documents: 688\n",
      "Number of processed documents: 689\n",
      "Number of processed documents: 690\n",
      "Number of processed documents: 691\n",
      "Number of processed documents: 692\n",
      "Number of processed documents: 693\n",
      "Number of processed documents: 694\n",
      "Number of processed documents: 695\n",
      "Number of processed documents: 696\n",
      "Number of processed documents: 697\n",
      "Number of processed documents: 698\n",
      "Number of processed documents: 699\n",
      "Number of processed documents: 700\n",
      "Number of processed documents: 701\n",
      "Number of processed documents: 702\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 703\n",
      "Number of processed documents: 704\n",
      "Number of processed documents: 705\n",
      "Number of processed documents: 706\n",
      "Number of processed documents: 707\n",
      "Number of processed documents: 708\n",
      "Number of processed documents: 709\n",
      "Number of processed documents: 710\n",
      "Number of processed documents: 711\n",
      "Number of processed documents: 712\n",
      "Number of processed documents: 713\n",
      "Number of processed documents: 714\n",
      "Number of processed documents: 715\n",
      "Number of processed documents: 716\n",
      "Number of processed documents: 717\n",
      "Number of processed documents: 718\n",
      "Number of processed documents: 719\n",
      "Number of processed documents: 720\n",
      "Number of processed documents: 721\n",
      "Number of processed documents: 722\n",
      "Number of processed documents: 723\n",
      "Number of processed documents: 724\n",
      "Number of processed documents: 725\n",
      "Number of processed documents: 726\n",
      "Number of processed documents: 727\n",
      "Number of processed documents: 728\n",
      "Number of processed documents: 729\n",
      "Number of processed documents: 730\n",
      "Number of processed documents: 731\n",
      "Number of processed documents: 732\n",
      "Number of processed documents: 733\n",
      "Number of processed documents: 734\n",
      "Number of processed documents: 735\n",
      "Number of processed documents: 736\n",
      "Number of processed documents: 737\n",
      "Number of processed documents: 738\n",
      "Number of processed documents: 739\n",
      "Number of processed documents: 740\n",
      "Number of processed documents: 741\n",
      "Number of processed documents: 742\n",
      "Number of processed documents: 743\n",
      "Number of processed documents: 744\n",
      "Number of processed documents: 745\n",
      "Number of processed documents: 746\n",
      "Number of processed documents: 747\n",
      "Number of processed documents: 748\n",
      "Number of processed documents: 749\n",
      "Number of processed documents: 750\n",
      "Number of processed documents: 751\n",
      "Number of processed documents: 752\n",
      "Number of processed documents: 753\n",
      "Number of processed documents: 754\n",
      "Number of processed documents: 755\n",
      "Number of processed documents: 756\n",
      "Number of processed documents: 757\n",
      "Number of processed documents: 758\n",
      "Number of processed documents: 759\n",
      "Number of processed documents: 760\n",
      "Number of processed documents: 761\n",
      "Number of processed documents: 762\n",
      "Number of processed documents: 763\n",
      "Number of processed documents: 764\n",
      "Number of processed documents: 765\n",
      "Number of processed documents: 766\n",
      "Number of processed documents: 767\n",
      "Number of processed documents: 768\n",
      "Number of processed documents: 769\n",
      "Number of processed documents: 770\n",
      "Number of processed documents: 771\n",
      "Number of processed documents: 772\n",
      "Number of processed documents: 773\n",
      "Number of processed documents: 774\n",
      "Number of processed documents: 775\n",
      "Number of processed documents: 776\n",
      "Number of processed documents: 777\n",
      "Number of processed documents: 778\n",
      "Number of processed documents: 779\n",
      "Number of processed documents: 780\n",
      "Number of processed documents: 781\n",
      "Number of processed documents: 782\n",
      "Number of processed documents: 783\n",
      "Number of processed documents: 784\n",
      "Number of processed documents: 785\n",
      "Number of processed documents: 786\n",
      "Number of processed documents: 787\n",
      "Number of processed documents: 788\n",
      "Number of processed documents: 789\n",
      "Number of processed documents: 790\n",
      "Number of processed documents: 791\n",
      "Number of processed documents: 792\n",
      "Number of processed documents: 793\n",
      "Number of processed documents: 794\n",
      "Number of processed documents: 795\n",
      "Number of processed documents: 796\n",
      "Number of processed documents: 797\n",
      "Number of processed documents: 798\n",
      "Number of processed documents: 799\n",
      "Number of processed documents: 800\n",
      "Number of processed documents: 801\n",
      "Number of processed documents: 802\n",
      "Number of processed documents: 803\n",
      "Number of processed documents: 804\n",
      "Number of processed documents: 805\n",
      "Number of processed documents: 806\n",
      "Number of processed documents: 807\n",
      "Number of processed documents: 808\n",
      "Number of processed documents: 809\n",
      "Number of processed documents: 810\n",
      "Number of processed documents: 811\n",
      "Number of processed documents: 812\n",
      "Number of processed documents: 813\n",
      "Number of processed documents: 814\n",
      "Number of processed documents: 815\n",
      "Number of processed documents: 816\n",
      "Number of processed documents: 817\n",
      "Number of processed documents: 818\n",
      "Number of processed documents: 819\n",
      "Number of processed documents: 820\n",
      "Number of processed documents: 821\n",
      "Number of processed documents: 822\n",
      "Number of processed documents: 823\n",
      "Number of processed documents: 824\n",
      "Number of processed documents: 825\n",
      "Number of processed documents: 826\n",
      "Number of processed documents: 827\n",
      "Number of processed documents: 828\n",
      "Number of processed documents: 829\n",
      "Number of processed documents: 830\n",
      "Number of processed documents: 831\n",
      "Number of processed documents: 832\n",
      "Number of processed documents: 833\n",
      "Number of processed documents: 834\n",
      "Number of processed documents: 835\n",
      "Number of processed documents: 836\n",
      "Number of processed documents: 837\n",
      "Number of processed documents: 838\n",
      "Number of processed documents: 839\n",
      "Number of processed documents: 840\n",
      "Number of processed documents: 841\n",
      "Number of processed documents: 842\n",
      "Number of processed documents: 843\n",
      "Number of processed documents: 844\n",
      "Number of processed documents: 845\n",
      "Number of processed documents: 846\n",
      "Number of processed documents: 847\n",
      "Number of processed documents: 848\n",
      "Number of processed documents: 849\n",
      "Number of processed documents: 850\n",
      "Number of processed documents: 851\n",
      "Number of processed documents: 852\n",
      "Number of processed documents: 853\n",
      "Number of processed documents: 854\n",
      "Number of processed documents: 855\n",
      "Number of processed documents: 856\n",
      "Number of processed documents: 857\n",
      "Number of processed documents: 858\n",
      "Number of processed documents: 859\n",
      "Number of processed documents: 860\n",
      "Number of processed documents: 861\n",
      "Number of processed documents: 862\n",
      "Number of processed documents: 863\n",
      "Number of processed documents: 864\n",
      "Number of processed documents: 865\n",
      "Number of processed documents: 866\n",
      "Number of processed documents: 867\n",
      "Number of processed documents: 868\n",
      "Number of processed documents: 869\n",
      "Number of processed documents: 870\n",
      "Number of processed documents: 871\n",
      "Number of processed documents: 872\n",
      "Number of processed documents: 873\n",
      "Number of processed documents: 874\n",
      "Number of processed documents: 875\n",
      "Number of processed documents: 876\n",
      "Number of processed documents: 877\n",
      "Number of processed documents: 878\n",
      "Number of processed documents: 879\n",
      "Number of processed documents: 880\n",
      "Number of processed documents: 881\n",
      "Number of processed documents: 882\n",
      "Number of processed documents: 883\n",
      "Number of processed documents: 884\n",
      "Number of processed documents: 885\n",
      "Number of processed documents: 886\n",
      "Number of processed documents: 887\n",
      "Number of processed documents: 888\n",
      "Number of processed documents: 889\n",
      "Number of processed documents: 890\n",
      "Number of processed documents: 891\n",
      "Number of processed documents: 892\n",
      "Number of processed documents: 893\n",
      "Number of processed documents: 894\n",
      "Number of processed documents: 895\n",
      "Number of processed documents: 896\n",
      "Number of processed documents: 897\n",
      "Number of processed documents: 898\n",
      "Number of processed documents: 899\n",
      "Number of processed documents: 900\n",
      "Number of processed documents: 901\n",
      "Number of processed documents: 902\n",
      "Number of processed documents: 903\n",
      "Number of processed documents: 904\n",
      "Number of processed documents: 905\n",
      "Number of processed documents: 906\n",
      "Number of processed documents: 907\n",
      "Number of processed documents: 908\n",
      "Number of processed documents: 909\n",
      "Number of processed documents: 910\n",
      "Number of processed documents: 911\n",
      "Number of processed documents: 912\n",
      "Number of processed documents: 913\n",
      "Number of processed documents: 914\n",
      "Number of processed documents: 915\n",
      "Number of processed documents: 916\n",
      "Number of processed documents: 917\n",
      "Number of processed documents: 918\n",
      "Number of processed documents: 919\n",
      "Number of processed documents: 920\n",
      "Number of processed documents: 921\n",
      "Number of processed documents: 922\n",
      "Number of processed documents: 923\n",
      "Number of processed documents: 924\n",
      "Number of processed documents: 925\n",
      "Number of processed documents: 926\n",
      "Number of processed documents: 927\n",
      "Number of processed documents: 928\n",
      "Number of processed documents: 929\n",
      "Number of processed documents: 930\n",
      "Number of processed documents: 931\n",
      "Number of processed documents: 932\n",
      "Number of processed documents: 933\n",
      "Number of processed documents: 934\n",
      "Number of processed documents: 935\n",
      "Number of processed documents: 936\n",
      "Number of processed documents: 937\n",
      "Number of processed documents: 938\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 939\n",
      "Number of processed documents: 940\n",
      "Number of processed documents: 941\n",
      "Number of processed documents: 942\n",
      "Number of processed documents: 943\n",
      "Number of processed documents: 944\n",
      "Number of processed documents: 945\n",
      "Number of processed documents: 946\n",
      "Number of processed documents: 947\n",
      "Number of processed documents: 948\n",
      "Number of processed documents: 949\n",
      "Number of processed documents: 950\n",
      "Number of processed documents: 951\n",
      "Number of processed documents: 952\n",
      "Number of processed documents: 953\n",
      "Number of processed documents: 954\n",
      "Number of processed documents: 955\n",
      "Number of processed documents: 956\n",
      "Number of processed documents: 957\n",
      "Number of processed documents: 958\n",
      "Number of processed documents: 959\n",
      "Number of processed documents: 960\n",
      "Number of processed documents: 961\n",
      "Number of processed documents: 962\n",
      "Number of processed documents: 963\n",
      "Number of processed documents: 964\n",
      "Number of processed documents: 965\n",
      "Number of processed documents: 966\n",
      "Number of processed documents: 967\n",
      "Number of processed documents: 968\n",
      "Number of processed documents: 969\n",
      "Number of processed documents: 970\n",
      "Number of processed documents: 971\n",
      "Number of processed documents: 972\n",
      "Number of processed documents: 973\n",
      "Number of processed documents: 974\n",
      "Number of processed documents: 975\n",
      "Number of processed documents: 976\n",
      "Number of processed documents: 977\n",
      "Number of processed documents: 978\n",
      "Number of processed documents: 979\n",
      "Number of processed documents: 980\n",
      "Number of processed documents: 981\n",
      "Number of processed documents: 982\n",
      "Number of processed documents: 983\n",
      "Number of processed documents: 984\n",
      "Number of processed documents: 985\n",
      "Number of processed documents: 986\n",
      "Number of processed documents: 987\n",
      "Number of processed documents: 988\n",
      "Number of processed documents: 989\n",
      "Number of processed documents: 990\n",
      "Number of processed documents: 991\n",
      "Number of processed documents: 992\n",
      "Number of processed documents: 993\n",
      "Number of processed documents: 994\n",
      "Number of processed documents: 995\n",
      "Number of processed documents: 996\n",
      "Number of processed documents: 997\n",
      "Number of processed documents: 998\n",
      "Number of processed documents: 999\n",
      "Number of processed documents: 1000\n",
      "Number of processed documents: 1001\n",
      "Number of processed documents: 1002\n",
      "Number of processed documents: 1003\n",
      "Number of processed documents: 1004\n",
      "Number of processed documents: 1005\n",
      "Number of processed documents: 1006\n",
      "Number of processed documents: 1007\n",
      "Number of processed documents: 1008\n",
      "Number of processed documents: 1009\n",
      "Number of processed documents: 1010\n",
      "Number of processed documents: 1011\n",
      "Number of processed documents: 1012\n",
      "Number of processed documents: 1013\n",
      "Number of processed documents: 1014\n",
      "Number of processed documents: 1015\n",
      "Number of processed documents: 1016\n",
      "Number of processed documents: 1017\n",
      "Number of processed documents: 1018\n",
      "Number of processed documents: 1019\n",
      "Number of processed documents: 1020\n",
      "Number of processed documents: 1021\n",
      "Number of processed documents: 1022\n",
      "Number of processed documents: 1023\n",
      "Number of processed documents: 1024\n",
      "Number of processed documents: 1025\n",
      "Number of processed documents: 1026\n",
      "Number of processed documents: 1027\n",
      "Number of processed documents: 1028\n",
      "Number of processed documents: 1029\n",
      "Number of processed documents: 1030\n",
      "Number of processed documents: 1031\n",
      "Number of processed documents: 1032\n",
      "Number of processed documents: 1033\n",
      "Number of processed documents: 1034\n",
      "Number of processed documents: 1035\n",
      "Number of processed documents: 1036\n",
      "Number of processed documents: 1037\n",
      "Number of processed documents: 1038\n",
      "Number of processed documents: 1039\n",
      "Number of processed documents: 1040\n",
      "Number of processed documents: 1041\n",
      "Number of processed documents: 1042\n",
      "Number of processed documents: 1043\n",
      "Number of processed documents: 1044\n",
      "Number of processed documents: 1045\n",
      "Number of processed documents: 1046\n",
      "Number of processed documents: 1047\n",
      "Number of processed documents: 1048\n",
      "Number of processed documents: 1049\n",
      "Number of processed documents: 1050\n",
      "Number of processed documents: 1051\n",
      "Number of processed documents: 1052\n",
      "Number of processed documents: 1053\n",
      "Number of processed documents: 1054\n",
      "Number of processed documents: 1055\n",
      "Number of processed documents: 1056\n",
      "Number of processed documents: 1057\n",
      "Number of processed documents: 1058\n",
      "Number of processed documents: 1059\n",
      "Number of processed documents: 1060\n",
      "Number of processed documents: 1061\n",
      "Number of processed documents: 1062\n",
      "Number of processed documents: 1063\n",
      "Number of processed documents: 1064\n",
      "Number of processed documents: 1065\n",
      "Number of processed documents: 1066\n",
      "Number of processed documents: 1067\n",
      "Number of processed documents: 1068\n",
      "Number of processed documents: 1069\n",
      "Number of processed documents: 1070\n",
      "Number of processed documents: 1071\n",
      "Number of processed documents: 1072\n",
      "Number of processed documents: 1073\n",
      "Number of processed documents: 1074\n",
      "Number of processed documents: 1075\n",
      "Number of processed documents: 1076\n",
      "Number of processed documents: 1077\n",
      "Number of processed documents: 1078\n",
      "Number of processed documents: 1079\n",
      "Number of processed documents: 1080\n",
      "Number of processed documents: 1081\n",
      "Number of processed documents: 1082\n",
      "Number of processed documents: 1083\n",
      "Number of processed documents: 1084\n",
      "Number of processed documents: 1085\n",
      "Number of processed documents: 1086\n",
      "Number of processed documents: 1087\n",
      "Number of processed documents: 1088\n",
      "Number of processed documents: 1089\n",
      "Number of processed documents: 1090\n",
      "Number of processed documents: 1091\n",
      "Number of processed documents: 1092\n",
      "Number of processed documents: 1093\n",
      "Number of processed documents: 1094\n",
      "Number of processed documents: 1095\n",
      "Number of processed documents: 1096\n",
      "Number of processed documents: 1097\n",
      "Number of processed documents: 1098\n",
      "Number of processed documents: 1099\n",
      "Number of processed documents: 1100\n",
      "Number of processed documents: 1101\n",
      "Number of processed documents: 1102\n",
      "Number of processed documents: 1103\n",
      "Number of processed documents: 1104\n",
      "Number of processed documents: 1105\n",
      "Number of processed documents: 1106\n",
      "Number of processed documents: 1107\n",
      "Number of processed documents: 1108\n",
      "Number of processed documents: 1109\n",
      "Number of processed documents: 1110\n",
      "Number of processed documents: 1111\n",
      "Number of processed documents: 1112\n",
      "Number of processed documents: 1113\n",
      "Number of processed documents: 1114\n",
      "Number of processed documents: 1115\n",
      "Number of processed documents: 1116\n",
      "Number of processed documents: 1117\n",
      "Number of processed documents: 1118\n",
      "Number of processed documents: 1119\n",
      "Number of processed documents: 1120\n",
      "Number of processed documents: 1121\n",
      "Number of processed documents: 1122\n",
      "Number of processed documents: 1123\n",
      "Number of processed documents: 1124\n",
      "Number of processed documents: 1125\n",
      "Number of processed documents: 1126\n",
      "Number of processed documents: 1127\n",
      "Number of processed documents: 1128\n",
      "Number of processed documents: 1129\n",
      "Number of processed documents: 1130\n",
      "Number of processed documents: 1131\n",
      "Number of processed documents: 1132\n",
      "Number of processed documents: 1133\n",
      "Number of processed documents: 1134\n",
      "Number of processed documents: 1135\n",
      "Number of processed documents: 1136\n",
      "Number of processed documents: 1137\n",
      "Number of processed documents: 1138\n",
      "Number of processed documents: 1139\n",
      "Number of processed documents: 1140\n",
      "Number of processed documents: 1141\n",
      "Number of processed documents: 1142\n",
      "Number of processed documents: 1143\n",
      "Number of processed documents: 1144\n",
      "Number of processed documents: 1145\n",
      "Number of processed documents: 1146\n",
      "Number of processed documents: 1147\n",
      "Number of processed documents: 1148\n",
      "Number of processed documents: 1149\n",
      "Number of processed documents: 1150\n",
      "Number of processed documents: 1151\n",
      "Number of processed documents: 1152\n",
      "Number of processed documents: 1153\n",
      "Number of processed documents: 1154\n",
      "Number of processed documents: 1155\n",
      "Number of processed documents: 1156\n",
      "Number of processed documents: 1157\n",
      "Number of processed documents: 1158\n",
      "Number of processed documents: 1159\n",
      "Number of processed documents: 1160\n",
      "Number of processed documents: 1161\n",
      "Number of processed documents: 1162\n",
      "Number of processed documents: 1163\n",
      "Number of processed documents: 1164\n",
      "Number of processed documents: 1165\n",
      "Number of processed documents: 1166\n",
      "Number of processed documents: 1167\n",
      "Number of processed documents: 1168\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 1169\n",
      "Number of processed documents: 1170\n",
      "Number of processed documents: 1171\n",
      "Number of processed documents: 1172\n",
      "Number of processed documents: 1173\n",
      "Number of processed documents: 1174\n",
      "Number of processed documents: 1175\n",
      "Number of processed documents: 1176\n",
      "Number of processed documents: 1177\n",
      "Number of processed documents: 1178\n",
      "Number of processed documents: 1179\n",
      "Number of processed documents: 1180\n",
      "Number of processed documents: 1181\n",
      "Number of processed documents: 1182\n",
      "Number of processed documents: 1183\n",
      "Number of processed documents: 1184\n",
      "Number of processed documents: 1185\n",
      "Number of processed documents: 1186\n",
      "Number of processed documents: 1187\n",
      "Number of processed documents: 1188\n",
      "Number of processed documents: 1189\n",
      "Number of processed documents: 1190\n",
      "Number of processed documents: 1191\n",
      "Number of processed documents: 1192\n",
      "Number of processed documents: 1193\n",
      "Number of processed documents: 1194\n",
      "Number of processed documents: 1195\n",
      "Number of processed documents: 1196\n",
      "Number of processed documents: 1197\n",
      "Number of processed documents: 1198\n",
      "Number of processed documents: 1199\n",
      "Number of processed documents: 1200\n",
      "Number of processed documents: 1201\n",
      "Number of processed documents: 1202\n",
      "Number of processed documents: 1203\n",
      "Number of processed documents: 1204\n",
      "Number of processed documents: 1205\n",
      "Number of processed documents: 1206\n",
      "Number of processed documents: 1207\n",
      "Number of processed documents: 1208\n",
      "Number of processed documents: 1209\n",
      "Number of processed documents: 1210\n",
      "Number of processed documents: 1211\n",
      "Number of processed documents: 1212\n",
      "Number of processed documents: 1213\n",
      "Number of processed documents: 1214\n",
      "Number of processed documents: 1215\n",
      "Number of processed documents: 1216\n",
      "Number of processed documents: 1217\n",
      "Number of processed documents: 1218\n",
      "Number of processed documents: 1219\n",
      "Number of processed documents: 1220\n",
      "Number of processed documents: 1221\n",
      "Number of processed documents: 1222\n",
      "Number of processed documents: 1223\n",
      "Number of processed documents: 1224\n",
      "Number of processed documents: 1225\n",
      "Number of processed documents: 1226\n",
      "Number of processed documents: 1227\n",
      "Number of processed documents: 1228\n",
      "Number of processed documents: 1229\n",
      "Number of processed documents: 1230\n",
      "Number of processed documents: 1231\n",
      "Number of processed documents: 1232\n",
      "Number of processed documents: 1233\n",
      "Number of processed documents: 1234\n",
      "Number of processed documents: 1235\n",
      "Number of processed documents: 1236\n",
      "Number of processed documents: 1237\n",
      "Number of processed documents: 1238\n",
      "Number of processed documents: 1239\n",
      "Number of processed documents: 1240\n",
      "Number of processed documents: 1241\n",
      "Number of processed documents: 1242\n",
      "Number of processed documents: 1243\n",
      "Number of processed documents: 1244\n",
      "Number of processed documents: 1245\n",
      "Number of processed documents: 1246\n",
      "Number of processed documents: 1247\n",
      "Number of processed documents: 1248\n",
      "Number of processed documents: 1249\n",
      "Number of processed documents: 1250\n",
      "Number of processed documents: 1251\n",
      "Number of processed documents: 1252\n",
      "Number of processed documents: 1253\n",
      "Number of processed documents: 1254\n",
      "Number of processed documents: 1255\n",
      "Number of processed documents: 1256\n",
      "Number of processed documents: 1257\n",
      "Number of processed documents: 1258\n",
      "Number of processed documents: 1259\n",
      "Number of processed documents: 1260\n",
      "Number of processed documents: 1261\n",
      "Number of processed documents: 1262\n",
      "Number of processed documents: 1263\n",
      "Number of processed documents: 1264\n",
      "Number of processed documents: 1265\n",
      "Number of processed documents: 1266\n",
      "Number of processed documents: 1267\n",
      "Number of processed documents: 1268\n",
      "Number of processed documents: 1269\n",
      "Number of processed documents: 1270\n",
      "Number of processed documents: 1271\n",
      "Number of processed documents: 1272\n",
      "Number of processed documents: 1273\n",
      "Number of processed documents: 1274\n",
      "Number of processed documents: 1275\n",
      "Number of processed documents: 1276\n",
      "Number of processed documents: 1277\n",
      "Number of processed documents: 1278\n",
      "Number of processed documents: 1279\n",
      "Number of processed documents: 1280\n",
      "Number of processed documents: 1281\n",
      "Number of processed documents: 1282\n",
      "Number of processed documents: 1283\n",
      "Number of processed documents: 1284\n",
      "Number of processed documents: 1285\n",
      "Number of processed documents: 1286\n",
      "Number of processed documents: 1287\n",
      "Number of processed documents: 1288\n",
      "Number of processed documents: 1289\n",
      "Number of processed documents: 1290\n",
      "Number of processed documents: 1291\n",
      "Number of processed documents: 1292\n",
      "Number of processed documents: 1293\n",
      "Number of processed documents: 1294\n",
      "Number of processed documents: 1295\n",
      "Number of processed documents: 1296\n",
      "Number of processed documents: 1297\n",
      "Number of processed documents: 1298\n",
      "Number of processed documents: 1299\n",
      "Number of processed documents: 1300\n",
      "Number of processed documents: 1301\n",
      "Number of processed documents: 1302\n",
      "Number of processed documents: 1303\n",
      "Number of processed documents: 1304\n",
      "Number of processed documents: 1305\n",
      "Number of processed documents: 1306\n",
      "Number of processed documents: 1307\n",
      "Number of processed documents: 1308\n",
      "Number of processed documents: 1309\n",
      "Number of processed documents: 1310\n",
      "Number of processed documents: 1311\n",
      "Number of processed documents: 1312\n",
      "Number of processed documents: 1313\n",
      "Number of processed documents: 1314\n",
      "Number of processed documents: 1315\n",
      "Number of processed documents: 1316\n",
      "Number of processed documents: 1317\n",
      "Number of processed documents: 1318\n",
      "Number of processed documents: 1319\n",
      "Number of processed documents: 1320\n",
      "Number of processed documents: 1321\n",
      "Number of processed documents: 1322\n",
      "Number of processed documents: 1323\n",
      "Number of processed documents: 1324\n",
      "Number of processed documents: 1325\n",
      "Number of processed documents: 1326\n",
      "Number of processed documents: 1327\n",
      "Number of processed documents: 1328\n",
      "Number of processed documents: 1329\n",
      "Number of processed documents: 1330\n",
      "Number of processed documents: 1331\n",
      "Number of processed documents: 1332\n",
      "Number of processed documents: 1333\n",
      "Number of processed documents: 1334\n",
      "Number of processed documents: 1335\n",
      "Number of processed documents: 1336\n",
      "Number of processed documents: 1337\n",
      "Number of processed documents: 1338\n",
      "Number of processed documents: 1339\n",
      "Number of processed documents: 1340\n",
      "Number of processed documents: 1341\n",
      "Number of processed documents: 1342\n",
      "Number of processed documents: 1343\n",
      "Number of processed documents: 1344\n",
      "Number of processed documents: 1345\n",
      "Number of processed documents: 1346\n",
      "Number of processed documents: 1347\n",
      "Number of processed documents: 1348\n",
      "Number of processed documents: 1349\n",
      "Number of processed documents: 1350\n",
      "Number of processed documents: 1351\n",
      "Number of processed documents: 1352\n",
      "Number of processed documents: 1353\n",
      "Number of processed documents: 1354\n",
      "Number of processed documents: 1355\n",
      "Number of processed documents: 1356\n",
      "Number of processed documents: 1357\n",
      "Number of processed documents: 1358\n",
      "Number of processed documents: 1359\n",
      "Number of processed documents: 1360\n",
      "Number of processed documents: 1361\n",
      "Number of processed documents: 1362\n",
      "Number of processed documents: 1363\n",
      "Number of processed documents: 1364\n",
      "Number of processed documents: 1365\n",
      "Number of processed documents: 1366\n",
      "Number of processed documents: 1367\n",
      "Number of processed documents: 1368\n",
      "Number of processed documents: 1369\n",
      "Number of processed documents: 1370\n",
      "Number of processed documents: 1371\n",
      "Number of processed documents: 1372\n",
      "Number of processed documents: 1373\n",
      "Number of processed documents: 1374\n",
      "Number of processed documents: 1375\n",
      "Number of processed documents: 1376\n",
      "Number of processed documents: 1377\n",
      "Number of processed documents: 1378\n",
      "Number of processed documents: 1379\n",
      "Number of processed documents: 1380\n",
      "Number of processed documents: 1381\n",
      "Number of processed documents: 1382\n",
      "Number of processed documents: 1383\n",
      "Number of processed documents: 1384\n",
      "Number of processed documents: 1385\n",
      "Number of processed documents: 1386\n",
      "Number of processed documents: 1387\n",
      "Number of processed documents: 1388\n",
      "Number of processed documents: 1389\n",
      "Number of processed documents: 1390\n",
      "Number of processed documents: 1391\n",
      "Number of processed documents: 1392\n",
      "Number of processed documents: 1393\n",
      "Number of processed documents: 1394\n",
      "Number of processed documents: 1395\n",
      "Number of processed documents: 1396\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 1397\n",
      "Number of processed documents: 1398\n",
      "Number of processed documents: 1399\n",
      "Number of processed documents: 1400\n",
      "Number of processed documents: 1401\n",
      "Number of processed documents: 1402\n",
      "Number of processed documents: 1403\n",
      "Number of processed documents: 1404\n",
      "Number of processed documents: 1405\n",
      "Number of processed documents: 1406\n",
      "Number of processed documents: 1407\n",
      "Number of processed documents: 1408\n",
      "Number of processed documents: 1409\n",
      "Number of processed documents: 1410\n",
      "Number of processed documents: 1411\n",
      "Number of processed documents: 1412\n",
      "Number of processed documents: 1413\n",
      "Number of processed documents: 1414\n",
      "Number of processed documents: 1415\n",
      "Number of processed documents: 1416\n",
      "Number of processed documents: 1417\n",
      "Number of processed documents: 1418\n",
      "Number of processed documents: 1419\n",
      "Number of processed documents: 1420\n",
      "Number of processed documents: 1421\n",
      "Number of processed documents: 1422\n",
      "Number of processed documents: 1423\n",
      "Number of processed documents: 1424\n",
      "Number of processed documents: 1425\n",
      "Number of processed documents: 1426\n",
      "Number of processed documents: 1427\n",
      "Number of processed documents: 1428\n",
      "Number of processed documents: 1429\n",
      "Number of processed documents: 1430\n",
      "Number of processed documents: 1431\n",
      "Number of processed documents: 1432\n",
      "Number of processed documents: 1433\n",
      "Number of processed documents: 1434\n",
      "Number of processed documents: 1435\n",
      "Number of processed documents: 1436\n",
      "Number of processed documents: 1437\n",
      "Number of processed documents: 1438\n",
      "Number of processed documents: 1439\n",
      "Number of processed documents: 1440\n",
      "Number of processed documents: 1441\n",
      "Number of processed documents: 1442\n",
      "Number of processed documents: 1443\n",
      "Number of processed documents: 1444\n",
      "Number of processed documents: 1445\n",
      "Number of processed documents: 1446\n",
      "Number of processed documents: 1447\n",
      "Number of processed documents: 1448\n",
      "Number of processed documents: 1449\n",
      "Number of processed documents: 1450\n",
      "Number of processed documents: 1451\n",
      "Number of processed documents: 1452\n",
      "Number of processed documents: 1453\n",
      "Number of processed documents: 1454\n",
      "Number of processed documents: 1455\n",
      "Number of processed documents: 1456\n",
      "Number of processed documents: 1457\n",
      "Number of processed documents: 1458\n",
      "Number of processed documents: 1459\n",
      "Number of processed documents: 1460\n",
      "Number of processed documents: 1461\n",
      "Number of processed documents: 1462\n",
      "Number of processed documents: 1463\n",
      "Number of processed documents: 1464\n",
      "Number of processed documents: 1465\n",
      "Number of processed documents: 1466\n",
      "Number of processed documents: 1467\n",
      "Number of processed documents: 1468\n",
      "Number of processed documents: 1469\n",
      "Number of processed documents: 1470\n",
      "Number of processed documents: 1471\n",
      "Number of processed documents: 1472\n",
      "Number of processed documents: 1473\n",
      "Number of processed documents: 1474\n",
      "Number of processed documents: 1475\n",
      "Number of processed documents: 1476\n",
      "Number of processed documents: 1477\n",
      "Number of processed documents: 1478\n",
      "Number of processed documents: 1479\n",
      "Number of processed documents: 1480\n",
      "Number of processed documents: 1481\n",
      "Number of processed documents: 1482\n",
      "Number of processed documents: 1483\n",
      "Number of processed documents: 1484\n",
      "Number of processed documents: 1485\n",
      "Number of processed documents: 1486\n",
      "Number of processed documents: 1487\n",
      "Number of processed documents: 1488\n",
      "Number of processed documents: 1489\n",
      "Number of processed documents: 1490\n",
      "Number of processed documents: 1491\n",
      "Number of processed documents: 1492\n",
      "Number of processed documents: 1493\n",
      "Number of processed documents: 1494\n",
      "Number of processed documents: 1495\n",
      "Number of processed documents: 1496\n",
      "Number of processed documents: 1497\n",
      "Number of processed documents: 1498\n",
      "Number of processed documents: 1499\n",
      "Number of processed documents: 1500\n",
      "Number of processed documents: 1501\n",
      "Number of processed documents: 1502\n",
      "Number of processed documents: 1503\n",
      "Number of processed documents: 1504\n",
      "Number of processed documents: 1505\n",
      "Number of processed documents: 1506\n",
      "Number of processed documents: 1507\n",
      "Number of processed documents: 1508\n",
      "Number of processed documents: 1509\n",
      "Number of processed documents: 1510\n",
      "Number of processed documents: 1511\n",
      "Number of processed documents: 1512\n",
      "Number of processed documents: 1513\n",
      "Number of processed documents: 1514\n",
      "Number of processed documents: 1515\n",
      "Number of processed documents: 1516\n",
      "Number of processed documents: 1517\n",
      "Number of processed documents: 1518\n",
      "Number of processed documents: 1519\n",
      "Number of processed documents: 1520\n",
      "Number of processed documents: 1521\n",
      "Number of processed documents: 1522\n",
      "Number of processed documents: 1523\n",
      "Number of processed documents: 1524\n",
      "Number of processed documents: 1525\n",
      "Number of processed documents: 1526\n",
      "Number of processed documents: 1527\n",
      "Number of processed documents: 1528\n",
      "Number of processed documents: 1529\n",
      "Number of processed documents: 1530\n",
      "Number of processed documents: 1531\n",
      "Number of processed documents: 1532\n",
      "Number of processed documents: 1533\n",
      "Number of processed documents: 1534\n",
      "Number of processed documents: 1535\n",
      "Number of processed documents: 1536\n",
      "Number of processed documents: 1537\n",
      "Number of processed documents: 1538\n",
      "Number of processed documents: 1539\n",
      "Number of processed documents: 1540\n",
      "Number of processed documents: 1541\n",
      "Number of processed documents: 1542\n",
      "Number of processed documents: 1543\n",
      "Number of processed documents: 1544\n",
      "Number of processed documents: 1545\n",
      "Number of processed documents: 1546\n",
      "Number of processed documents: 1547\n",
      "Number of processed documents: 1548\n",
      "Number of processed documents: 1549\n",
      "Number of processed documents: 1550\n",
      "Number of processed documents: 1551\n",
      "Number of processed documents: 1552\n",
      "Number of processed documents: 1553\n",
      "Number of processed documents: 1554\n",
      "Number of processed documents: 1555\n",
      "Number of processed documents: 1556\n",
      "Number of processed documents: 1557\n",
      "Number of processed documents: 1558\n",
      "Number of processed documents: 1559\n",
      "Number of processed documents: 1560\n",
      "Number of processed documents: 1561\n",
      "Number of processed documents: 1562\n",
      "Number of processed documents: 1563\n",
      "Number of processed documents: 1564\n",
      "Number of processed documents: 1565\n",
      "Number of processed documents: 1566\n",
      "Number of processed documents: 1567\n",
      "Number of processed documents: 1568\n",
      "Number of processed documents: 1569\n",
      "Number of processed documents: 1570\n",
      "Number of processed documents: 1571\n",
      "Number of processed documents: 1572\n",
      "Number of processed documents: 1573\n",
      "Number of processed documents: 1574\n",
      "Number of processed documents: 1575\n",
      "Number of processed documents: 1576\n",
      "Number of processed documents: 1577\n",
      "Number of processed documents: 1578\n",
      "Number of processed documents: 1579\n",
      "Number of processed documents: 1580\n",
      "Number of processed documents: 1581\n",
      "Number of processed documents: 1582\n",
      "Number of processed documents: 1583\n",
      "Number of processed documents: 1584\n",
      "Number of processed documents: 1585\n",
      "Number of processed documents: 1586\n",
      "Number of processed documents: 1587\n",
      "Number of processed documents: 1588\n",
      "Number of processed documents: 1589\n",
      "Number of processed documents: 1590\n",
      "Number of processed documents: 1591\n",
      "Number of processed documents: 1592\n",
      "Number of processed documents: 1593\n",
      "Number of processed documents: 1594\n",
      "Number of processed documents: 1595\n",
      "Number of processed documents: 1596\n",
      "Number of processed documents: 1597\n",
      "Number of processed documents: 1598\n",
      "Number of processed documents: 1599\n",
      "Number of processed documents: 1600\n",
      "Number of processed documents: 1601\n",
      "Number of processed documents: 1602\n",
      "Number of processed documents: 1603\n",
      "Number of processed documents: 1604\n",
      "Number of processed documents: 1605\n",
      "Number of processed documents: 1606\n",
      "Number of processed documents: 1607\n",
      "Number of processed documents: 1608\n",
      "Number of processed documents: 1609\n",
      "Number of processed documents: 1610\n",
      "Number of processed documents: 1611\n",
      "Number of processed documents: 1612\n",
      "Number of processed documents: 1613\n",
      "Number of processed documents: 1614\n",
      "Number of processed documents: 1615\n",
      "Number of processed documents: 1616\n",
      "Number of processed documents: 1617\n",
      "Number of processed documents: 1618\n",
      "Number of processed documents: 1619\n",
      "Number of processed documents: 1620\n",
      "Number of processed documents: 1621\n",
      "Number of processed documents: 1622\n",
      "Number of processed documents: 1623\n",
      "Number of processed documents: 1624\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 1625\n",
      "Number of processed documents: 1626\n",
      "Number of processed documents: 1627\n",
      "Number of processed documents: 1628\n",
      "Number of processed documents: 1629\n",
      "Number of processed documents: 1630\n",
      "Number of processed documents: 1631\n",
      "Number of processed documents: 1632\n",
      "Number of processed documents: 1633\n",
      "Number of processed documents: 1634\n",
      "Number of processed documents: 1635\n",
      "Number of processed documents: 1636\n",
      "Number of processed documents: 1637\n",
      "Number of processed documents: 1638\n",
      "Number of processed documents: 1639\n",
      "Number of processed documents: 1640\n",
      "Number of processed documents: 1641\n",
      "Number of processed documents: 1642\n",
      "Number of processed documents: 1643\n",
      "Number of processed documents: 1644\n",
      "Number of processed documents: 1645\n",
      "Number of processed documents: 1646\n",
      "Number of processed documents: 1647\n",
      "Number of processed documents: 1648\n",
      "Number of processed documents: 1649\n",
      "Number of processed documents: 1650\n",
      "Number of processed documents: 1651\n",
      "Number of processed documents: 1652\n",
      "Number of processed documents: 1653\n",
      "Number of processed documents: 1654\n",
      "Number of processed documents: 1655\n",
      "Number of processed documents: 1656\n",
      "Number of processed documents: 1657\n",
      "Number of processed documents: 1658\n",
      "Number of processed documents: 1659\n",
      "Number of processed documents: 1660\n",
      "Number of processed documents: 1661\n",
      "Number of processed documents: 1662\n",
      "Number of processed documents: 1663\n",
      "Number of processed documents: 1664\n",
      "Number of processed documents: 1665\n",
      "Number of processed documents: 1666\n",
      "Number of processed documents: 1667\n",
      "Number of processed documents: 1668\n",
      "Number of processed documents: 1669\n",
      "Number of processed documents: 1670\n",
      "Number of processed documents: 1671\n",
      "Number of processed documents: 1672\n",
      "Number of processed documents: 1673\n",
      "Number of processed documents: 1674\n",
      "Number of processed documents: 1675\n",
      "Number of processed documents: 1676\n",
      "Number of processed documents: 1677\n",
      "Number of processed documents: 1678\n",
      "Number of processed documents: 1679\n",
      "Number of processed documents: 1680\n",
      "Number of processed documents: 1681\n",
      "Number of processed documents: 1682\n",
      "Number of processed documents: 1683\n",
      "Number of processed documents: 1684\n",
      "Number of processed documents: 1685\n",
      "Number of processed documents: 1686\n",
      "Number of processed documents: 1687\n",
      "Number of processed documents: 1688\n",
      "Number of processed documents: 1689\n",
      "Number of processed documents: 1690\n",
      "Number of processed documents: 1691\n",
      "Number of processed documents: 1692\n",
      "Number of processed documents: 1693\n",
      "Number of processed documents: 1694\n",
      "Number of processed documents: 1695\n",
      "Number of processed documents: 1696\n",
      "Number of processed documents: 1697\n",
      "Number of processed documents: 1698\n",
      "Number of processed documents: 1699\n",
      "Number of processed documents: 1700\n",
      "Number of processed documents: 1701\n",
      "Number of processed documents: 1702\n",
      "Number of processed documents: 1703\n",
      "Number of processed documents: 1704\n",
      "Number of processed documents: 1705\n",
      "Number of processed documents: 1706\n",
      "Number of processed documents: 1707\n",
      "Number of processed documents: 1708\n",
      "Number of processed documents: 1709\n",
      "Number of processed documents: 1710\n",
      "Number of processed documents: 1711\n",
      "Number of processed documents: 1712\n",
      "Number of processed documents: 1713\n",
      "Number of processed documents: 1714\n",
      "Number of processed documents: 1715\n",
      "Number of processed documents: 1716\n",
      "Number of processed documents: 1717\n",
      "Number of processed documents: 1718\n",
      "Number of processed documents: 1719\n",
      "Number of processed documents: 1720\n",
      "Number of processed documents: 1721\n",
      "Number of processed documents: 1722\n",
      "Number of processed documents: 1723\n",
      "Number of processed documents: 1724\n",
      "Number of processed documents: 1725\n",
      "Number of processed documents: 1726\n",
      "Number of processed documents: 1727\n",
      "Number of processed documents: 1728\n",
      "Number of processed documents: 1729\n",
      "Number of processed documents: 1730\n",
      "Number of processed documents: 1731\n",
      "Number of processed documents: 1732\n",
      "Number of processed documents: 1733\n",
      "Number of processed documents: 1734\n",
      "Number of processed documents: 1735\n",
      "Number of processed documents: 1736\n",
      "Number of processed documents: 1737\n",
      "Number of processed documents: 1738\n",
      "Number of processed documents: 1739\n",
      "Number of processed documents: 1740\n",
      "Number of processed documents: 1741\n",
      "Number of processed documents: 1742\n",
      "Number of processed documents: 1743\n",
      "Number of processed documents: 1744\n",
      "Number of processed documents: 1745\n",
      "Number of processed documents: 1746\n",
      "Number of processed documents: 1747\n",
      "Number of processed documents: 1748\n",
      "Number of processed documents: 1749\n",
      "Number of processed documents: 1750\n",
      "Number of processed documents: 1751\n",
      "Number of processed documents: 1752\n",
      "Number of processed documents: 1753\n",
      "Number of processed documents: 1754\n",
      "Number of processed documents: 1755\n",
      "Number of processed documents: 1756\n",
      "Number of processed documents: 1757\n",
      "Number of processed documents: 1758\n",
      "Number of processed documents: 1759\n",
      "Number of processed documents: 1760\n",
      "Number of processed documents: 1761\n",
      "Number of processed documents: 1762\n",
      "Number of processed documents: 1763\n",
      "Number of processed documents: 1764\n",
      "Number of processed documents: 1765\n",
      "Number of processed documents: 1766\n",
      "Number of processed documents: 1767\n",
      "Number of processed documents: 1768\n",
      "Number of processed documents: 1769\n",
      "Number of processed documents: 1770\n",
      "Number of processed documents: 1771\n",
      "Number of processed documents: 1772\n",
      "Number of processed documents: 1773\n",
      "Number of processed documents: 1774\n",
      "Number of processed documents: 1775\n",
      "Number of processed documents: 1776\n",
      "Number of processed documents: 1777\n",
      "Number of processed documents: 1778\n",
      "Number of processed documents: 1779\n",
      "Number of processed documents: 1780\n",
      "Number of processed documents: 1781\n",
      "Number of processed documents: 1782\n",
      "Number of processed documents: 1783\n",
      "Number of processed documents: 1784\n",
      "Number of processed documents: 1785\n",
      "Number of processed documents: 1786\n",
      "Number of processed documents: 1787\n",
      "Number of processed documents: 1788\n",
      "Number of processed documents: 1789\n",
      "Number of processed documents: 1790\n",
      "Number of processed documents: 1791\n",
      "Number of processed documents: 1792\n",
      "Number of processed documents: 1793\n",
      "Number of processed documents: 1794\n",
      "Number of processed documents: 1795\n",
      "Number of processed documents: 1796\n",
      "Number of processed documents: 1797\n",
      "Number of processed documents: 1798\n",
      "Number of processed documents: 1799\n",
      "Number of processed documents: 1800\n",
      "Number of processed documents: 1801\n",
      "Number of processed documents: 1802\n",
      "Number of processed documents: 1803\n",
      "Number of processed documents: 1804\n",
      "Number of processed documents: 1805\n",
      "Number of processed documents: 1806\n",
      "Number of processed documents: 1807\n",
      "Number of processed documents: 1808\n",
      "Number of processed documents: 1809\n",
      "Number of processed documents: 1810\n",
      "Number of processed documents: 1811\n",
      "Number of processed documents: 1812\n",
      "Number of processed documents: 1813\n",
      "Number of processed documents: 1814\n",
      "Number of processed documents: 1815\n",
      "Number of processed documents: 1816\n",
      "Number of processed documents: 1817\n",
      "Number of processed documents: 1818\n",
      "Number of processed documents: 1819\n",
      "Number of processed documents: 1820\n",
      "Number of processed documents: 1821\n",
      "Number of processed documents: 1822\n",
      "Number of processed documents: 1823\n",
      "Number of processed documents: 1824\n",
      "Number of processed documents: 1825\n",
      "Number of processed documents: 1826\n",
      "Number of processed documents: 1827\n",
      "Number of processed documents: 1828\n",
      "Number of processed documents: 1829\n",
      "Number of processed documents: 1830\n",
      "Number of processed documents: 1831\n",
      "Number of processed documents: 1832\n",
      "Number of processed documents: 1833\n",
      "Number of processed documents: 1834\n",
      "Number of processed documents: 1835\n",
      "Number of processed documents: 1836\n",
      "Number of processed documents: 1837\n",
      "Number of processed documents: 1838\n",
      "Number of processed documents: 1839\n",
      "Number of processed documents: 1840\n",
      "Number of processed documents: 1841\n",
      "Number of processed documents: 1842\n",
      "Number of processed documents: 1843\n",
      "Number of processed documents: 1844\n",
      "Number of processed documents: 1845\n",
      "Number of processed documents: 1846\n",
      "Number of processed documents: 1847\n",
      "Number of processed documents: 1848\n",
      "Number of processed documents: 1849\n",
      "Number of processed documents: 1850\n",
      "Number of processed documents: 1851\n",
      "Number of processed documents: 1852\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 1853\n",
      "Number of processed documents: 1854\n",
      "Number of processed documents: 1855\n",
      "Number of processed documents: 1856\n",
      "Number of processed documents: 1857\n",
      "Number of processed documents: 1858\n",
      "Number of processed documents: 1859\n",
      "Number of processed documents: 1860\n",
      "Number of processed documents: 1861\n",
      "Number of processed documents: 1862\n",
      "Number of processed documents: 1863\n",
      "Number of processed documents: 1864\n",
      "Number of processed documents: 1865\n",
      "Number of processed documents: 1866\n",
      "Number of processed documents: 1867\n",
      "Number of processed documents: 1868\n",
      "Number of processed documents: 1869\n",
      "Number of processed documents: 1870\n",
      "Number of processed documents: 1871\n",
      "Number of processed documents: 1872\n",
      "Number of processed documents: 1873\n",
      "Number of processed documents: 1874\n",
      "Number of processed documents: 1875\n",
      "Number of processed documents: 1876\n",
      "Number of processed documents: 1877\n",
      "Number of processed documents: 1878\n",
      "Number of processed documents: 1879\n",
      "Number of processed documents: 1880\n",
      "Number of processed documents: 1881\n",
      "Number of processed documents: 1882\n",
      "Number of processed documents: 1883\n",
      "Number of processed documents: 1884\n",
      "Number of processed documents: 1885\n",
      "Number of processed documents: 1886\n",
      "Number of processed documents: 1887\n",
      "Number of processed documents: 1888\n",
      "Number of processed documents: 1889\n",
      "Number of processed documents: 1890\n",
      "Number of processed documents: 1891\n",
      "Number of processed documents: 1892\n",
      "Number of processed documents: 1893\n",
      "Number of processed documents: 1894\n",
      "Number of processed documents: 1895\n",
      "Number of processed documents: 1896\n",
      "Number of processed documents: 1897\n",
      "Number of processed documents: 1898\n",
      "Number of processed documents: 1899\n",
      "Number of processed documents: 1900\n",
      "Number of processed documents: 1901\n",
      "Number of processed documents: 1902\n",
      "Number of processed documents: 1903\n",
      "Number of processed documents: 1904\n",
      "Number of processed documents: 1905\n",
      "Number of processed documents: 1906\n",
      "Number of processed documents: 1907\n",
      "Number of processed documents: 1908\n",
      "Number of processed documents: 1909\n",
      "Number of processed documents: 1910\n",
      "Number of processed documents: 1911\n",
      "Number of processed documents: 1912\n",
      "Number of processed documents: 1913\n",
      "Number of processed documents: 1914\n",
      "Number of processed documents: 1915\n",
      "Number of processed documents: 1916\n",
      "Number of processed documents: 1917\n",
      "Number of processed documents: 1918\n",
      "Number of processed documents: 1919\n",
      "Number of processed documents: 1920\n",
      "Number of processed documents: 1921\n",
      "Number of processed documents: 1922\n",
      "Number of processed documents: 1923\n",
      "Number of processed documents: 1924\n",
      "Number of processed documents: 1925\n",
      "Number of processed documents: 1926\n",
      "Number of processed documents: 1927\n",
      "Number of processed documents: 1928\n",
      "Number of processed documents: 1929\n",
      "Number of processed documents: 1930\n",
      "Number of processed documents: 1931\n",
      "Number of processed documents: 1932\n",
      "Number of processed documents: 1933\n",
      "Number of processed documents: 1934\n",
      "Number of processed documents: 1935\n",
      "Number of processed documents: 1936\n",
      "Number of processed documents: 1937\n",
      "Number of processed documents: 1938\n",
      "Number of processed documents: 1939\n",
      "Number of processed documents: 1940\n",
      "Number of processed documents: 1941\n",
      "Number of processed documents: 1942\n",
      "Number of processed documents: 1943\n",
      "Number of processed documents: 1944\n",
      "Number of processed documents: 1945\n",
      "Number of processed documents: 1946\n",
      "Number of processed documents: 1947\n",
      "Number of processed documents: 1948\n",
      "Number of processed documents: 1949\n",
      "Number of processed documents: 1950\n",
      "Number of processed documents: 1951\n",
      "Number of processed documents: 1952\n",
      "Number of processed documents: 1953\n",
      "Number of processed documents: 1954\n",
      "Number of processed documents: 1955\n",
      "Number of processed documents: 1956\n",
      "Number of processed documents: 1957\n",
      "Number of processed documents: 1958\n",
      "Number of processed documents: 1959\n",
      "Number of processed documents: 1960\n",
      "Number of processed documents: 1961\n",
      "Number of processed documents: 1962\n",
      "Number of processed documents: 1963\n",
      "Number of processed documents: 1964\n",
      "Number of processed documents: 1965\n",
      "Number of processed documents: 1966\n",
      "Number of processed documents: 1967\n",
      "Number of processed documents: 1968\n",
      "Number of processed documents: 1969\n",
      "Number of processed documents: 1970\n",
      "Number of processed documents: 1971\n",
      "Number of processed documents: 1972\n",
      "Number of processed documents: 1973\n",
      "Number of processed documents: 1974\n",
      "Number of processed documents: 1975\n",
      "Number of processed documents: 1976\n",
      "Number of processed documents: 1977\n",
      "Number of processed documents: 1978\n",
      "Number of processed documents: 1979\n",
      "Number of processed documents: 1980\n",
      "Number of processed documents: 1981\n",
      "Number of processed documents: 1982\n",
      "Number of processed documents: 1983\n",
      "Number of processed documents: 1984\n",
      "Number of processed documents: 1985\n",
      "Number of processed documents: 1986\n",
      "Number of processed documents: 1987\n",
      "Number of processed documents: 1988\n",
      "Number of processed documents: 1989\n",
      "Number of processed documents: 1990\n",
      "Number of processed documents: 1991\n",
      "Number of processed documents: 1992\n",
      "Number of processed documents: 1993\n",
      "Number of processed documents: 1994\n",
      "Number of processed documents: 1995\n",
      "Number of processed documents: 1996\n",
      "Number of processed documents: 1997\n",
      "Number of processed documents: 1998\n",
      "Number of processed documents: 1999\n",
      "Number of processed documents: 2000\n",
      "Number of processed documents: 2001\n",
      "Number of processed documents: 2002\n",
      "Number of processed documents: 2003\n",
      "Number of processed documents: 2004\n",
      "Number of processed documents: 2005\n",
      "Number of processed documents: 2006\n",
      "Number of processed documents: 2007\n",
      "Number of processed documents: 2008\n",
      "Number of processed documents: 2009\n",
      "Number of processed documents: 2010\n",
      "Number of processed documents: 2011\n",
      "Number of processed documents: 2012\n",
      "Number of processed documents: 2013\n",
      "Number of processed documents: 2014\n",
      "Number of processed documents: 2015\n",
      "Number of processed documents: 2016\n",
      "Number of processed documents: 2017\n",
      "Number of processed documents: 2018\n",
      "Number of processed documents: 2019\n",
      "Number of processed documents: 2020\n",
      "Number of processed documents: 2021\n",
      "Number of processed documents: 2022\n",
      "Number of processed documents: 2023\n",
      "Number of processed documents: 2024\n",
      "Number of processed documents: 2025\n",
      "Number of processed documents: 2026\n",
      "Number of processed documents: 2027\n",
      "Number of processed documents: 2028\n",
      "Number of processed documents: 2029\n",
      "Number of processed documents: 2030\n",
      "Number of processed documents: 2031\n",
      "Number of processed documents: 2032\n",
      "Number of processed documents: 2033\n",
      "Number of processed documents: 2034\n",
      "Number of processed documents: 2035\n",
      "Number of processed documents: 2036\n",
      "Number of processed documents: 2037\n",
      "Number of processed documents: 2038\n",
      "Number of processed documents: 2039\n",
      "Number of processed documents: 2040\n",
      "Number of processed documents: 2041\n",
      "Number of processed documents: 2042\n",
      "Number of processed documents: 2043\n",
      "Number of processed documents: 2044\n",
      "Number of processed documents: 2045\n",
      "Number of processed documents: 2046\n",
      "Number of processed documents: 2047\n",
      "Number of processed documents: 2048\n",
      "Number of processed documents: 2049\n",
      "Number of processed documents: 2050\n",
      "Number of processed documents: 2051\n",
      "Number of processed documents: 2052\n",
      "Number of processed documents: 2053\n",
      "Number of processed documents: 2054\n",
      "Number of processed documents: 2055\n",
      "Number of processed documents: 2056\n",
      "Number of processed documents: 2057\n",
      "Number of processed documents: 2058\n",
      "Number of processed documents: 2059\n",
      "Number of processed documents: 2060\n",
      "Number of processed documents: 2061\n",
      "Number of processed documents: 2062\n",
      "Number of processed documents: 2063\n",
      "Number of processed documents: 2064\n",
      "Number of processed documents: 2065\n",
      "Number of processed documents: 2066\n",
      "Number of processed documents: 2067\n",
      "Number of processed documents: 2068\n",
      "Number of processed documents: 2069\n",
      "Number of processed documents: 2070\n",
      "Number of processed documents: 2071\n",
      "Number of processed documents: 2072\n",
      "Number of processed documents: 2073\n",
      "Number of processed documents: 2074\n",
      "Number of processed documents: 2075\n",
      "Number of processed documents: 2076\n",
      "Number of processed documents: 2077\n",
      "Number of processed documents: 2078\n",
      "Number of processed documents: 2079\n",
      "Number of processed documents: 2080\n",
      "Number of processed documents: 2081\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 2082\n",
      "Number of processed documents: 2083\n",
      "Number of processed documents: 2084\n",
      "Number of processed documents: 2085\n",
      "Number of processed documents: 2086\n",
      "Number of processed documents: 2087\n",
      "Number of processed documents: 2088\n",
      "Number of processed documents: 2089\n",
      "Number of processed documents: 2090\n",
      "Number of processed documents: 2091\n",
      "Number of processed documents: 2092\n",
      "Number of processed documents: 2093\n",
      "Number of processed documents: 2094\n",
      "Number of processed documents: 2095\n",
      "Number of processed documents: 2096\n",
      "Number of processed documents: 2097\n",
      "Number of processed documents: 2098\n",
      "Number of processed documents: 2099\n",
      "Number of processed documents: 2100\n",
      "Number of processed documents: 2101\n",
      "Number of processed documents: 2102\n",
      "Number of processed documents: 2103\n",
      "Number of processed documents: 2104\n",
      "Number of processed documents: 2105\n",
      "Number of processed documents: 2106\n",
      "Number of processed documents: 2107\n",
      "Number of processed documents: 2108\n",
      "Number of processed documents: 2109\n",
      "Number of processed documents: 2110\n",
      "Number of processed documents: 2111\n",
      "Number of processed documents: 2112\n",
      "Number of processed documents: 2113\n",
      "Number of processed documents: 2114\n",
      "Number of processed documents: 2115\n",
      "Number of processed documents: 2116\n",
      "Number of processed documents: 2117\n",
      "Number of processed documents: 2118\n",
      "Number of processed documents: 2119\n",
      "Number of processed documents: 2120\n",
      "Number of processed documents: 2121\n",
      "Number of processed documents: 2122\n",
      "Number of processed documents: 2123\n",
      "Number of processed documents: 2124\n",
      "Number of processed documents: 2125\n",
      "Number of processed documents: 2126\n",
      "Number of processed documents: 2127\n",
      "Number of processed documents: 2128\n",
      "Number of processed documents: 2129\n",
      "Number of processed documents: 2130\n",
      "Number of processed documents: 2131\n",
      "Number of processed documents: 2132\n",
      "Number of processed documents: 2133\n",
      "Number of processed documents: 2134\n",
      "Number of processed documents: 2135\n",
      "Number of processed documents: 2136\n",
      "Number of processed documents: 2137\n",
      "Number of processed documents: 2138\n",
      "Number of processed documents: 2139\n",
      "Number of processed documents: 2140\n",
      "Number of processed documents: 2141\n",
      "Number of processed documents: 2142\n",
      "Number of processed documents: 2143\n",
      "Number of processed documents: 2144\n",
      "Number of processed documents: 2145\n",
      "Number of processed documents: 2146\n",
      "Number of processed documents: 2147\n",
      "Number of processed documents: 2148\n",
      "Number of processed documents: 2149\n",
      "Number of processed documents: 2150\n",
      "Number of processed documents: 2151\n",
      "Number of processed documents: 2152\n",
      "Number of processed documents: 2153\n",
      "Number of processed documents: 2154\n",
      "Number of processed documents: 2155\n",
      "Number of processed documents: 2156\n",
      "Number of processed documents: 2157\n",
      "Number of processed documents: 2158\n",
      "Number of processed documents: 2159\n",
      "Number of processed documents: 2160\n",
      "Number of processed documents: 2161\n",
      "Number of processed documents: 2162\n",
      "Number of processed documents: 2163\n",
      "Number of processed documents: 2164\n",
      "Number of processed documents: 2165\n",
      "Number of processed documents: 2166\n",
      "Number of processed documents: 2167\n",
      "Number of processed documents: 2168\n",
      "Number of processed documents: 2169\n",
      "Number of processed documents: 2170\n",
      "Number of processed documents: 2171\n",
      "Number of processed documents: 2172\n",
      "Number of processed documents: 2173\n",
      "Number of processed documents: 2174\n",
      "Number of processed documents: 2175\n",
      "Number of processed documents: 2176\n",
      "Number of processed documents: 2177\n",
      "Number of processed documents: 2178\n",
      "Number of processed documents: 2179\n",
      "Number of processed documents: 2180\n",
      "Number of processed documents: 2181\n",
      "Number of processed documents: 2182\n",
      "Number of processed documents: 2183\n",
      "Number of processed documents: 2184\n",
      "Number of processed documents: 2185\n",
      "Number of processed documents: 2186\n",
      "Number of processed documents: 2187\n",
      "Number of processed documents: 2188\n",
      "Number of processed documents: 2189\n",
      "Number of processed documents: 2190\n",
      "Number of processed documents: 2191\n",
      "Number of processed documents: 2192\n",
      "Number of processed documents: 2193\n",
      "Number of processed documents: 2194\n",
      "Number of processed documents: 2195\n",
      "Number of processed documents: 2196\n",
      "Number of processed documents: 2197\n",
      "Number of processed documents: 2198\n",
      "Number of processed documents: 2199\n",
      "Number of processed documents: 2200\n",
      "Number of processed documents: 2201\n",
      "Number of processed documents: 2202\n",
      "Number of processed documents: 2203\n",
      "Number of processed documents: 2204\n",
      "Number of processed documents: 2205\n",
      "Number of processed documents: 2206\n",
      "Number of processed documents: 2207\n",
      "Number of processed documents: 2208\n",
      "Number of processed documents: 2209\n",
      "Number of processed documents: 2210\n",
      "Number of processed documents: 2211\n",
      "Number of processed documents: 2212\n",
      "Number of processed documents: 2213\n",
      "Number of processed documents: 2214\n",
      "Number of processed documents: 2215\n",
      "Number of processed documents: 2216\n",
      "Number of processed documents: 2217\n",
      "Number of processed documents: 2218\n",
      "Number of processed documents: 2219\n",
      "Number of processed documents: 2220\n",
      "Number of processed documents: 2221\n",
      "Number of processed documents: 2222\n",
      "Number of processed documents: 2223\n",
      "Number of processed documents: 2224\n",
      "Number of processed documents: 2225\n",
      "Number of processed documents: 2226\n",
      "Number of processed documents: 2227\n",
      "Number of processed documents: 2228\n",
      "Number of processed documents: 2229\n",
      "Number of processed documents: 2230\n",
      "Number of processed documents: 2231\n",
      "Number of processed documents: 2232\n",
      "Number of processed documents: 2233\n",
      "Number of processed documents: 2234\n",
      "Number of processed documents: 2235\n",
      "Number of processed documents: 2236\n",
      "Number of processed documents: 2237\n",
      "Number of processed documents: 2238\n",
      "Number of processed documents: 2239\n",
      "Number of processed documents: 2240\n",
      "Number of processed documents: 2241\n",
      "Number of processed documents: 2242\n",
      "Number of processed documents: 2243\n",
      "Number of processed documents: 2244\n",
      "Number of processed documents: 2245\n",
      "Number of processed documents: 2246\n",
      "Number of processed documents: 2247\n",
      "Number of processed documents: 2248\n",
      "Number of processed documents: 2249\n",
      "Number of processed documents: 2250\n",
      "Number of processed documents: 2251\n",
      "Number of processed documents: 2252\n",
      "Number of processed documents: 2253\n",
      "Number of processed documents: 2254\n",
      "Number of processed documents: 2255\n",
      "Number of processed documents: 2256\n",
      "Number of processed documents: 2257\n",
      "Number of processed documents: 2258\n",
      "Number of processed documents: 2259\n",
      "Number of processed documents: 2260\n",
      "Number of processed documents: 2261\n",
      "Number of processed documents: 2262\n",
      "Number of processed documents: 2263\n",
      "Number of processed documents: 2264\n",
      "Number of processed documents: 2265\n",
      "Number of processed documents: 2266\n",
      "Number of processed documents: 2267\n",
      "Number of processed documents: 2268\n",
      "Number of processed documents: 2269\n",
      "Number of processed documents: 2270\n",
      "Number of processed documents: 2271\n",
      "Number of processed documents: 2272\n",
      "Number of processed documents: 2273\n",
      "Number of processed documents: 2274\n",
      "Number of processed documents: 2275\n",
      "Number of processed documents: 2276\n",
      "Number of processed documents: 2277\n",
      "Number of processed documents: 2278\n",
      "Number of processed documents: 2279\n",
      "Number of processed documents: 2280\n",
      "Number of processed documents: 2281\n",
      "Number of processed documents: 2282\n",
      "Number of processed documents: 2283\n",
      "Number of processed documents: 2284\n",
      "Number of processed documents: 2285\n",
      "Number of processed documents: 2286\n",
      "Number of processed documents: 2287\n",
      "Number of processed documents: 2288\n",
      "Number of processed documents: 2289\n",
      "Number of processed documents: 2290\n",
      "Number of processed documents: 2291\n",
      "Number of processed documents: 2292\n",
      "Number of processed documents: 2293\n",
      "Number of processed documents: 2294\n",
      "Number of processed documents: 2295\n",
      "Number of processed documents: 2296\n",
      "Number of processed documents: 2297\n",
      "Number of processed documents: 2298\n",
      "Number of processed documents: 2299\n",
      "Number of processed documents: 2300\n",
      "Number of processed documents: 2301\n",
      "Number of processed documents: 2302\n",
      "Number of processed documents: 2303\n",
      "Number of processed documents: 2304\n",
      "Number of processed documents: 2305\n",
      "Number of processed documents: 2306\n",
      "Number of processed documents: 2307\n",
      "Number of processed documents: 2308\n",
      "Number of processed documents: 2309\n",
      "Number of processed documents: 2310\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 2311\n",
      "Number of processed documents: 2312\n",
      "Number of processed documents: 2313\n",
      "Number of processed documents: 2314\n",
      "Number of processed documents: 2315\n",
      "Number of processed documents: 2316\n",
      "Number of processed documents: 2317\n",
      "Number of processed documents: 2318\n",
      "Number of processed documents: 2319\n",
      "Number of processed documents: 2320\n",
      "Number of processed documents: 2321\n",
      "Number of processed documents: 2322\n",
      "Number of processed documents: 2323\n",
      "Number of processed documents: 2324\n",
      "Number of processed documents: 2325\n",
      "Number of processed documents: 2326\n",
      "Number of processed documents: 2327\n",
      "Number of processed documents: 2328\n",
      "Number of processed documents: 2329\n",
      "Number of processed documents: 2330\n",
      "Number of processed documents: 2331\n",
      "Number of processed documents: 2332\n",
      "Number of processed documents: 2333\n",
      "Number of processed documents: 2334\n",
      "Number of processed documents: 2335\n",
      "Number of processed documents: 2336\n",
      "Number of processed documents: 2337\n",
      "Number of processed documents: 2338\n",
      "Number of processed documents: 2339\n",
      "Number of processed documents: 2340\n",
      "Number of processed documents: 2341\n",
      "Number of processed documents: 2342\n",
      "Number of processed documents: 2343\n",
      "Number of processed documents: 2344\n",
      "Number of processed documents: 2345\n",
      "Number of processed documents: 2346\n",
      "Number of processed documents: 2347\n",
      "Number of processed documents: 2348\n",
      "Number of processed documents: 2349\n",
      "Number of processed documents: 2350\n",
      "Number of processed documents: 2351\n",
      "Number of processed documents: 2352\n",
      "Number of processed documents: 2353\n",
      "Number of processed documents: 2354\n",
      "Number of processed documents: 2355\n",
      "Number of processed documents: 2356\n",
      "Number of processed documents: 2357\n",
      "Number of processed documents: 2358\n",
      "Number of processed documents: 2359\n",
      "Number of processed documents: 2360\n",
      "Number of processed documents: 2361\n",
      "Number of processed documents: 2362\n",
      "Number of processed documents: 2363\n",
      "Number of processed documents: 2364\n",
      "Number of processed documents: 2365\n",
      "Number of processed documents: 2366\n",
      "Number of processed documents: 2367\n",
      "Number of processed documents: 2368\n",
      "Number of processed documents: 2369\n",
      "Number of processed documents: 2370\n",
      "Number of processed documents: 2371\n",
      "Number of processed documents: 2372\n",
      "Number of processed documents: 2373\n",
      "Number of processed documents: 2374\n",
      "Number of processed documents: 2375\n",
      "Number of processed documents: 2376\n",
      "Number of processed documents: 2377\n",
      "Number of processed documents: 2378\n",
      "Number of processed documents: 2379\n",
      "Number of processed documents: 2380\n",
      "Number of processed documents: 2381\n",
      "Number of processed documents: 2382\n",
      "Number of processed documents: 2383\n",
      "Number of processed documents: 2384\n",
      "Number of processed documents: 2385\n",
      "Number of processed documents: 2386\n",
      "Number of processed documents: 2387\n",
      "Number of processed documents: 2388\n",
      "Number of processed documents: 2389\n",
      "Number of processed documents: 2390\n",
      "Number of processed documents: 2391\n",
      "Number of processed documents: 2392\n",
      "Number of processed documents: 2393\n",
      "Number of processed documents: 2394\n",
      "Number of processed documents: 2395\n",
      "Number of processed documents: 2396\n",
      "Number of processed documents: 2397\n",
      "Number of processed documents: 2398\n",
      "Number of processed documents: 2399\n",
      "Number of processed documents: 2400\n",
      "Number of processed documents: 2401\n",
      "Number of processed documents: 2402\n",
      "Number of processed documents: 2403\n",
      "Number of processed documents: 2404\n",
      "Number of processed documents: 2405\n",
      "Number of processed documents: 2406\n",
      "Number of processed documents: 2407\n",
      "Number of processed documents: 2408\n",
      "Number of processed documents: 2409\n",
      "Number of processed documents: 2410\n",
      "Number of processed documents: 2411\n",
      "Number of processed documents: 2412\n",
      "Number of processed documents: 2413\n",
      "Number of processed documents: 2414\n",
      "Number of processed documents: 2415\n",
      "Number of processed documents: 2416\n",
      "Number of processed documents: 2417\n",
      "Number of processed documents: 2418\n",
      "Number of processed documents: 2419\n",
      "Number of processed documents: 2420\n",
      "Number of processed documents: 2421\n",
      "Number of processed documents: 2422\n",
      "Number of processed documents: 2423\n",
      "Number of processed documents: 2424\n",
      "Number of processed documents: 2425\n",
      "Number of processed documents: 2426\n",
      "Number of processed documents: 2427\n",
      "Number of processed documents: 2428\n",
      "Number of processed documents: 2429\n",
      "Number of processed documents: 2430\n",
      "Number of processed documents: 2431\n",
      "Number of processed documents: 2432\n",
      "Number of processed documents: 2433\n",
      "Number of processed documents: 2434\n",
      "Number of processed documents: 2435\n",
      "Number of processed documents: 2436\n",
      "Number of processed documents: 2437\n",
      "Number of processed documents: 2438\n",
      "Number of processed documents: 2439\n",
      "Number of processed documents: 2440\n",
      "Number of processed documents: 2441\n",
      "Number of processed documents: 2442\n",
      "Number of processed documents: 2443\n",
      "Number of processed documents: 2444\n",
      "Number of processed documents: 2445\n",
      "Number of processed documents: 2446\n",
      "Number of processed documents: 2447\n",
      "Number of processed documents: 2448\n",
      "Number of processed documents: 2449\n",
      "Number of processed documents: 2450\n",
      "Number of processed documents: 2451\n",
      "Number of processed documents: 2452\n",
      "Number of processed documents: 2453\n",
      "Number of processed documents: 2454\n",
      "Number of processed documents: 2455\n",
      "Number of processed documents: 2456\n",
      "Number of processed documents: 2457\n",
      "Number of processed documents: 2458\n",
      "Number of processed documents: 2459\n",
      "Number of processed documents: 2460\n",
      "Number of processed documents: 2461\n",
      "Number of processed documents: 2462\n",
      "Number of processed documents: 2463\n",
      "Number of processed documents: 2464\n",
      "Number of processed documents: 2465\n",
      "Number of processed documents: 2466\n",
      "Number of processed documents: 2467\n",
      "Number of processed documents: 2468\n",
      "Number of processed documents: 2469\n",
      "Number of processed documents: 2470\n",
      "Number of processed documents: 2471\n",
      "Number of processed documents: 2472\n",
      "Number of processed documents: 2473\n",
      "Number of processed documents: 2474\n",
      "Number of processed documents: 2475\n",
      "Number of processed documents: 2476\n",
      "Number of processed documents: 2477\n",
      "Number of processed documents: 2478\n",
      "Number of processed documents: 2479\n",
      "Number of processed documents: 2480\n",
      "Number of processed documents: 2481\n",
      "Number of processed documents: 2482\n",
      "Number of processed documents: 2483\n",
      "Number of processed documents: 2484\n",
      "Number of processed documents: 2485\n",
      "Number of processed documents: 2486\n",
      "Number of processed documents: 2487\n",
      "Number of processed documents: 2488\n",
      "Number of processed documents: 2489\n",
      "Number of processed documents: 2490\n",
      "Number of processed documents: 2491\n",
      "Number of processed documents: 2492\n",
      "Number of processed documents: 2493\n",
      "Number of processed documents: 2494\n",
      "Number of processed documents: 2495\n",
      "Number of processed documents: 2496\n",
      "Number of processed documents: 2497\n",
      "Number of processed documents: 2498\n",
      "Number of processed documents: 2499\n",
      "Number of processed documents: 2500\n",
      "Number of processed documents: 2501\n",
      "Number of processed documents: 2502\n",
      "Number of processed documents: 2503\n",
      "Number of processed documents: 2504\n",
      "Number of processed documents: 2505\n",
      "Number of processed documents: 2506\n",
      "Number of processed documents: 2507\n",
      "Number of processed documents: 2508\n",
      "Number of processed documents: 2509\n",
      "Number of processed documents: 2510\n",
      "Number of processed documents: 2511\n",
      "Number of processed documents: 2512\n",
      "Number of processed documents: 2513\n",
      "Number of processed documents: 2514\n",
      "Number of processed documents: 2515\n",
      "Number of processed documents: 2516\n",
      "Number of processed documents: 2517\n",
      "Number of processed documents: 2518\n",
      "Number of processed documents: 2519\n",
      "Number of processed documents: 2520\n",
      "Number of processed documents: 2521\n",
      "Number of processed documents: 2522\n",
      "Number of processed documents: 2523\n",
      "Number of processed documents: 2524\n",
      "Number of processed documents: 2525\n",
      "Number of processed documents: 2526\n",
      "Number of processed documents: 2527\n",
      "Number of processed documents: 2528\n",
      "Number of processed documents: 2529\n",
      "Number of processed documents: 2530\n",
      "Number of processed documents: 2531\n",
      "Number of processed documents: 2532\n",
      "Number of processed documents: 2533\n",
      "Number of processed documents: 2534\n",
      "Number of processed documents: 2535\n",
      "Number of processed documents: 2536\n",
      "Number of processed documents: 2537\n",
      "Number of processed documents: 2538\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 2539\n",
      "Number of processed documents: 2540\n",
      "Number of processed documents: 2541\n",
      "Number of processed documents: 2542\n",
      "Number of processed documents: 2543\n",
      "Number of processed documents: 2544\n",
      "Number of processed documents: 2545\n",
      "Number of processed documents: 2546\n",
      "Number of processed documents: 2547\n",
      "Number of processed documents: 2548\n",
      "Number of processed documents: 2549\n",
      "Number of processed documents: 2550\n",
      "Number of processed documents: 2551\n",
      "Number of processed documents: 2552\n",
      "Number of processed documents: 2553\n",
      "Number of processed documents: 2554\n",
      "Number of processed documents: 2555\n",
      "Number of processed documents: 2556\n",
      "Number of processed documents: 2557\n",
      "Number of processed documents: 2558\n",
      "Number of processed documents: 2559\n",
      "Number of processed documents: 2560\n",
      "Number of processed documents: 2561\n",
      "Number of processed documents: 2562\n",
      "Number of processed documents: 2563\n",
      "Number of processed documents: 2564\n",
      "Number of processed documents: 2565\n",
      "Number of processed documents: 2566\n",
      "Number of processed documents: 2567\n",
      "Number of processed documents: 2568\n",
      "Number of processed documents: 2569\n",
      "Number of processed documents: 2570\n",
      "Number of processed documents: 2571\n",
      "Number of processed documents: 2572\n",
      "Number of processed documents: 2573\n",
      "Number of processed documents: 2574\n",
      "Number of processed documents: 2575\n",
      "Number of processed documents: 2576\n",
      "Number of processed documents: 2577\n",
      "Number of processed documents: 2578\n",
      "Number of processed documents: 2579\n",
      "Number of processed documents: 2580\n",
      "Number of processed documents: 2581\n",
      "Number of processed documents: 2582\n",
      "Number of processed documents: 2583\n",
      "Number of processed documents: 2584\n",
      "Number of processed documents: 2585\n",
      "Number of processed documents: 2586\n",
      "Number of processed documents: 2587\n",
      "Number of processed documents: 2588\n",
      "Number of processed documents: 2589\n",
      "Number of processed documents: 2590\n",
      "Number of processed documents: 2591\n",
      "Number of processed documents: 2592\n",
      "Number of processed documents: 2593\n",
      "Number of processed documents: 2594\n",
      "Number of processed documents: 2595\n",
      "Number of processed documents: 2596\n",
      "Number of processed documents: 2597\n",
      "Number of processed documents: 2598\n",
      "Number of processed documents: 2599\n",
      "Number of processed documents: 2600\n",
      "Number of processed documents: 2601\n",
      "Number of processed documents: 2602\n",
      "Number of processed documents: 2603\n",
      "Number of processed documents: 2604\n",
      "Number of processed documents: 2605\n",
      "Number of processed documents: 2606\n",
      "Number of processed documents: 2607\n",
      "Number of processed documents: 2608\n",
      "Number of processed documents: 2609\n",
      "Number of processed documents: 2610\n",
      "Number of processed documents: 2611\n",
      "Number of processed documents: 2612\n",
      "Number of processed documents: 2613\n",
      "Number of processed documents: 2614\n",
      "Number of processed documents: 2615\n",
      "Number of processed documents: 2616\n",
      "Number of processed documents: 2617\n",
      "Number of processed documents: 2618\n",
      "Number of processed documents: 2619\n",
      "Number of processed documents: 2620\n",
      "Number of processed documents: 2621\n",
      "Number of processed documents: 2622\n",
      "Number of processed documents: 2623\n",
      "Number of processed documents: 2624\n",
      "Number of processed documents: 2625\n",
      "Number of processed documents: 2626\n",
      "Number of processed documents: 2627\n",
      "Number of processed documents: 2628\n",
      "Number of processed documents: 2629\n",
      "Number of processed documents: 2630\n",
      "Number of processed documents: 2631\n",
      "Number of processed documents: 2632\n",
      "Number of processed documents: 2633\n",
      "Number of processed documents: 2634\n",
      "Number of processed documents: 2635\n",
      "Number of processed documents: 2636\n",
      "Number of processed documents: 2637\n",
      "Number of processed documents: 2638\n",
      "Number of processed documents: 2639\n",
      "Number of processed documents: 2640\n",
      "Number of processed documents: 2641\n",
      "Number of processed documents: 2642\n",
      "Number of processed documents: 2643\n",
      "Number of processed documents: 2644\n",
      "Number of processed documents: 2645\n",
      "Number of processed documents: 2646\n",
      "Number of processed documents: 2647\n",
      "Number of processed documents: 2648\n",
      "Number of processed documents: 2649\n",
      "Number of processed documents: 2650\n",
      "Number of processed documents: 2651\n",
      "Number of processed documents: 2652\n",
      "Number of processed documents: 2653\n",
      "Number of processed documents: 2654\n",
      "Number of processed documents: 2655\n",
      "Number of processed documents: 2656\n",
      "Number of processed documents: 2657\n",
      "Number of processed documents: 2658\n",
      "Number of processed documents: 2659\n",
      "Number of processed documents: 2660\n",
      "Number of processed documents: 2661\n",
      "Number of processed documents: 2662\n",
      "Number of processed documents: 2663\n",
      "Number of processed documents: 2664\n",
      "Number of processed documents: 2665\n",
      "Number of processed documents: 2666\n",
      "Number of processed documents: 2667\n",
      "Number of processed documents: 2668\n",
      "Number of processed documents: 2669\n",
      "Number of processed documents: 2670\n",
      "Number of processed documents: 2671\n",
      "Number of processed documents: 2672\n",
      "Number of processed documents: 2673\n",
      "Number of processed documents: 2674\n",
      "Number of processed documents: 2675\n",
      "Number of processed documents: 2676\n",
      "Number of processed documents: 2677\n",
      "Number of processed documents: 2678\n",
      "Number of processed documents: 2679\n",
      "Number of processed documents: 2680\n",
      "Number of processed documents: 2681\n",
      "Number of processed documents: 2682\n",
      "Number of processed documents: 2683\n",
      "Number of processed documents: 2684\n",
      "Number of processed documents: 2685\n",
      "Number of processed documents: 2686\n",
      "Number of processed documents: 2687\n",
      "Number of processed documents: 2688\n",
      "Number of processed documents: 2689\n",
      "Number of processed documents: 2690\n",
      "Number of processed documents: 2691\n",
      "Number of processed documents: 2692\n",
      "Number of processed documents: 2693\n",
      "Number of processed documents: 2694\n",
      "Number of processed documents: 2695\n",
      "Number of processed documents: 2696\n",
      "Number of processed documents: 2697\n",
      "Number of processed documents: 2698\n",
      "Number of processed documents: 2699\n",
      "Number of processed documents: 2700\n",
      "Number of processed documents: 2701\n",
      "Number of processed documents: 2702\n",
      "Number of processed documents: 2703\n",
      "Number of processed documents: 2704\n",
      "Number of processed documents: 2705\n",
      "Number of processed documents: 2706\n",
      "Number of processed documents: 2707\n",
      "Number of processed documents: 2708\n",
      "Number of processed documents: 2709\n",
      "Number of processed documents: 2710\n",
      "Number of processed documents: 2711\n",
      "Number of processed documents: 2712\n",
      "Number of processed documents: 2713\n",
      "Number of processed documents: 2714\n",
      "Number of processed documents: 2715\n",
      "Number of processed documents: 2716\n",
      "Number of processed documents: 2717\n",
      "Number of processed documents: 2718\n",
      "Number of processed documents: 2719\n",
      "Number of processed documents: 2720\n",
      "Number of processed documents: 2721\n",
      "Number of processed documents: 2722\n",
      "Number of processed documents: 2723\n",
      "Number of processed documents: 2724\n",
      "Number of processed documents: 2725\n",
      "Number of processed documents: 2726\n",
      "Number of processed documents: 2727\n",
      "Number of processed documents: 2728\n",
      "Number of processed documents: 2729\n",
      "Number of processed documents: 2730\n",
      "Number of processed documents: 2731\n",
      "Number of processed documents: 2732\n",
      "Number of processed documents: 2733\n",
      "Number of processed documents: 2734\n",
      "Number of processed documents: 2735\n",
      "Number of processed documents: 2736\n",
      "Number of processed documents: 2737\n",
      "Number of processed documents: 2738\n",
      "Number of processed documents: 2739\n",
      "Number of processed documents: 2740\n",
      "Number of processed documents: 2741\n",
      "Number of processed documents: 2742\n",
      "Number of processed documents: 2743\n",
      "Number of processed documents: 2744\n",
      "Number of processed documents: 2745\n",
      "Number of processed documents: 2746\n",
      "Number of processed documents: 2747\n",
      "Number of processed documents: 2748\n",
      "Number of processed documents: 2749\n",
      "Number of processed documents: 2750\n",
      "Number of processed documents: 2751\n",
      "Number of processed documents: 2752\n",
      "Number of processed documents: 2753\n",
      "Number of processed documents: 2754\n",
      "Number of processed documents: 2755\n",
      "Number of processed documents: 2756\n",
      "Number of processed documents: 2757\n",
      "Number of processed documents: 2758\n",
      "Number of processed documents: 2759\n",
      "Number of processed documents: 2760\n",
      "Number of processed documents: 2761\n",
      "Number of processed documents: 2762\n",
      "Number of processed documents: 2763\n",
      "Number of processed documents: 2764\n",
      "Number of processed documents: 2765\n",
      "Number of processed documents: 2766\n",
      "Number of processed documents: 2767\n",
      "Number of processed documents: 2768\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 2769\n",
      "Number of processed documents: 2770\n",
      "Number of processed documents: 2771\n",
      "Number of processed documents: 2772\n",
      "Number of processed documents: 2773\n",
      "Number of processed documents: 2774\n",
      "Number of processed documents: 2775\n",
      "Number of processed documents: 2776\n",
      "Number of processed documents: 2777\n",
      "Number of processed documents: 2778\n",
      "Number of processed documents: 2779\n",
      "Number of processed documents: 2780\n",
      "Number of processed documents: 2781\n",
      "Number of processed documents: 2782\n",
      "Number of processed documents: 2783\n",
      "Number of processed documents: 2784\n",
      "Number of processed documents: 2785\n",
      "Number of processed documents: 2786\n",
      "Number of processed documents: 2787\n",
      "Number of processed documents: 2788\n",
      "Number of processed documents: 2789\n",
      "Number of processed documents: 2790\n",
      "Number of processed documents: 2791\n",
      "Number of processed documents: 2792\n",
      "Number of processed documents: 2793\n",
      "Number of processed documents: 2794\n",
      "Number of processed documents: 2795\n",
      "Number of processed documents: 2796\n",
      "Number of processed documents: 2797\n",
      "Number of processed documents: 2798\n",
      "Number of processed documents: 2799\n",
      "Number of processed documents: 2800\n",
      "Number of processed documents: 2801\n",
      "Number of processed documents: 2802\n",
      "Number of processed documents: 2803\n",
      "Number of processed documents: 2804\n",
      "Number of processed documents: 2805\n",
      "Number of processed documents: 2806\n",
      "Number of processed documents: 2807\n",
      "Number of processed documents: 2808\n",
      "Number of processed documents: 2809\n",
      "Number of processed documents: 2810\n",
      "Number of processed documents: 2811\n",
      "Number of processed documents: 2812\n",
      "Number of processed documents: 2813\n",
      "Number of processed documents: 2814\n",
      "Number of processed documents: 2815\n",
      "Number of processed documents: 2816\n",
      "Number of processed documents: 2817\n",
      "Number of processed documents: 2818\n",
      "Number of processed documents: 2819\n",
      "Number of processed documents: 2820\n",
      "Number of processed documents: 2821\n",
      "Number of processed documents: 2822\n",
      "Number of processed documents: 2823\n",
      "Number of processed documents: 2824\n",
      "Number of processed documents: 2825\n",
      "Number of processed documents: 2826\n",
      "Number of processed documents: 2827\n",
      "Number of processed documents: 2828\n",
      "Number of processed documents: 2829\n",
      "Number of processed documents: 2830\n",
      "Number of processed documents: 2831\n",
      "Number of processed documents: 2832\n",
      "Number of processed documents: 2833\n",
      "Number of processed documents: 2834\n",
      "Number of processed documents: 2835\n",
      "Number of processed documents: 2836\n",
      "Number of processed documents: 2837\n",
      "Number of processed documents: 2838\n",
      "Number of processed documents: 2839\n",
      "Number of processed documents: 2840\n",
      "Number of processed documents: 2841\n",
      "Number of processed documents: 2842\n",
      "Number of processed documents: 2843\n",
      "Number of processed documents: 2844\n",
      "Number of processed documents: 2845\n",
      "Number of processed documents: 2846\n",
      "Number of processed documents: 2847\n",
      "Number of processed documents: 2848\n",
      "Number of processed documents: 2849\n",
      "Number of processed documents: 2850\n",
      "Number of processed documents: 2851\n",
      "Number of processed documents: 2852\n",
      "Number of processed documents: 2853\n",
      "Number of processed documents: 2854\n",
      "Number of processed documents: 2855\n",
      "Number of processed documents: 2856\n",
      "Number of processed documents: 2857\n",
      "Number of processed documents: 2858\n",
      "Number of processed documents: 2859\n",
      "Number of processed documents: 2860\n",
      "Number of processed documents: 2861\n",
      "Number of processed documents: 2862\n",
      "Number of processed documents: 2863\n",
      "Number of processed documents: 2864\n",
      "Number of processed documents: 2865\n",
      "Number of processed documents: 2866\n",
      "Number of processed documents: 2867\n",
      "Number of processed documents: 2868\n",
      "Number of processed documents: 2869\n",
      "Number of processed documents: 2870\n",
      "Number of processed documents: 2871\n",
      "Number of processed documents: 2872\n",
      "Number of processed documents: 2873\n",
      "Number of processed documents: 2874\n",
      "Number of processed documents: 2875\n",
      "Number of processed documents: 2876\n",
      "Number of processed documents: 2877\n",
      "Number of processed documents: 2878\n",
      "Number of processed documents: 2879\n",
      "Number of processed documents: 2880\n",
      "Number of processed documents: 2881\n",
      "Number of processed documents: 2882\n",
      "Number of processed documents: 2883\n",
      "Number of processed documents: 2884\n",
      "Number of processed documents: 2885\n",
      "Number of processed documents: 2886\n",
      "Number of processed documents: 2887\n",
      "Number of processed documents: 2888\n",
      "Number of processed documents: 2889\n",
      "Number of processed documents: 2890\n",
      "Number of processed documents: 2891\n",
      "Number of processed documents: 2892\n",
      "Number of processed documents: 2893\n",
      "Number of processed documents: 2894\n",
      "Number of processed documents: 2895\n",
      "Number of processed documents: 2896\n",
      "Number of processed documents: 2897\n",
      "Number of processed documents: 2898\n",
      "Number of processed documents: 2899\n",
      "Number of processed documents: 2900\n",
      "Number of processed documents: 2901\n",
      "Number of processed documents: 2902\n",
      "Number of processed documents: 2903\n",
      "Number of processed documents: 2904\n",
      "Number of processed documents: 2905\n",
      "Number of processed documents: 2906\n",
      "Number of processed documents: 2907\n",
      "Number of processed documents: 2908\n",
      "Number of processed documents: 2909\n",
      "Number of processed documents: 2910\n",
      "Number of processed documents: 2911\n",
      "Number of processed documents: 2912\n",
      "Number of processed documents: 2913\n",
      "Number of processed documents: 2914\n",
      "Number of processed documents: 2915\n",
      "Number of processed documents: 2916\n",
      "Number of processed documents: 2917\n",
      "Number of processed documents: 2918\n",
      "Number of processed documents: 2919\n",
      "Number of processed documents: 2920\n",
      "Number of processed documents: 2921\n",
      "Number of processed documents: 2922\n",
      "Number of processed documents: 2923\n",
      "Number of processed documents: 2924\n",
      "Number of processed documents: 2925\n",
      "Number of processed documents: 2926\n",
      "Number of processed documents: 2927\n",
      "Number of processed documents: 2928\n",
      "Number of processed documents: 2929\n",
      "Number of processed documents: 2930\n",
      "Number of processed documents: 2931\n",
      "Number of processed documents: 2932\n",
      "Number of processed documents: 2933\n",
      "Number of processed documents: 2934\n",
      "Number of processed documents: 2935\n",
      "Number of processed documents: 2936\n",
      "Number of processed documents: 2937\n",
      "Number of processed documents: 2938\n",
      "Number of processed documents: 2939\n",
      "Number of processed documents: 2940\n",
      "Number of processed documents: 2941\n",
      "Number of processed documents: 2942\n",
      "Number of processed documents: 2943\n",
      "Number of processed documents: 2944\n",
      "Number of processed documents: 2945\n",
      "Number of processed documents: 2946\n",
      "Number of processed documents: 2947\n",
      "Number of processed documents: 2948\n",
      "Number of processed documents: 2949\n",
      "Number of processed documents: 2950\n",
      "Number of processed documents: 2951\n",
      "Number of processed documents: 2952\n",
      "Number of processed documents: 2953\n",
      "Number of processed documents: 2954\n",
      "Number of processed documents: 2955\n",
      "Number of processed documents: 2956\n",
      "Number of processed documents: 2957\n",
      "Number of processed documents: 2958\n",
      "Number of processed documents: 2959\n",
      "Number of processed documents: 2960\n",
      "Number of processed documents: 2961\n",
      "Number of processed documents: 2962\n",
      "Number of processed documents: 2963\n",
      "Number of processed documents: 2964\n",
      "Number of processed documents: 2965\n",
      "Number of processed documents: 2966\n",
      "Number of processed documents: 2967\n",
      "Number of processed documents: 2968\n",
      "Number of processed documents: 2969\n",
      "Number of processed documents: 2970\n",
      "Number of processed documents: 2971\n",
      "Number of processed documents: 2972\n",
      "Number of processed documents: 2973\n",
      "Number of processed documents: 2974\n",
      "Number of processed documents: 2975\n",
      "Number of processed documents: 2976\n",
      "Number of processed documents: 2977\n",
      "Number of processed documents: 2978\n",
      "Number of processed documents: 2979\n",
      "Number of processed documents: 2980\n",
      "Number of processed documents: 2981\n",
      "Number of processed documents: 2982\n",
      "Number of processed documents: 2983\n",
      "Number of processed documents: 2984\n",
      "Number of processed documents: 2985\n",
      "Number of processed documents: 2986\n",
      "Number of processed documents: 2987\n",
      "Number of processed documents: 2988\n",
      "Number of processed documents: 2989\n",
      "Number of processed documents: 2990\n",
      "Number of processed documents: 2991\n",
      "Number of processed documents: 2992\n",
      "Number of processed documents: 2993\n",
      "Number of processed documents: 2994\n",
      "Number of processed documents: 2995\n",
      "Number of processed documents: 2996\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 2997\n",
      "Number of processed documents: 2998\n",
      "Number of processed documents: 2999\n",
      "Number of processed documents: 3000\n",
      "Number of processed documents: 3001\n",
      "Number of processed documents: 3002\n",
      "Number of processed documents: 3003\n",
      "Number of processed documents: 3004\n",
      "Number of processed documents: 3005\n",
      "Number of processed documents: 3006\n",
      "Number of processed documents: 3007\n",
      "Number of processed documents: 3008\n",
      "Number of processed documents: 3009\n",
      "Number of processed documents: 3010\n",
      "Number of processed documents: 3011\n",
      "Number of processed documents: 3012\n",
      "Number of processed documents: 3013\n",
      "Number of processed documents: 3014\n",
      "Number of processed documents: 3015\n",
      "Number of processed documents: 3016\n",
      "Number of processed documents: 3017\n",
      "Number of processed documents: 3018\n",
      "Number of processed documents: 3019\n",
      "Number of processed documents: 3020\n",
      "Number of processed documents: 3021\n",
      "Number of processed documents: 3022\n",
      "Number of processed documents: 3023\n",
      "Number of processed documents: 3024\n",
      "Number of processed documents: 3025\n",
      "Number of processed documents: 3026\n",
      "Number of processed documents: 3027\n",
      "Number of processed documents: 3028\n",
      "Number of processed documents: 3029\n",
      "Number of processed documents: 3030\n",
      "Number of processed documents: 3031\n",
      "Number of processed documents: 3032\n",
      "Number of processed documents: 3033\n",
      "Number of processed documents: 3034\n",
      "Number of processed documents: 3035\n",
      "Number of processed documents: 3036\n",
      "Number of processed documents: 3037\n",
      "Number of processed documents: 3038\n",
      "Number of processed documents: 3039\n",
      "Number of processed documents: 3040\n",
      "Number of processed documents: 3041\n",
      "Number of processed documents: 3042\n",
      "Number of processed documents: 3043\n",
      "Number of processed documents: 3044\n",
      "Number of processed documents: 3045\n",
      "Number of processed documents: 3046\n",
      "Number of processed documents: 3047\n",
      "Number of processed documents: 3048\n",
      "Number of processed documents: 3049\n",
      "Number of processed documents: 3050\n",
      "Number of processed documents: 3051\n",
      "Number of processed documents: 3052\n",
      "Number of processed documents: 3053\n",
      "Number of processed documents: 3054\n",
      "Number of processed documents: 3055\n",
      "Number of processed documents: 3056\n",
      "Number of processed documents: 3057\n",
      "Number of processed documents: 3058\n",
      "Number of processed documents: 3059\n",
      "Number of processed documents: 3060\n",
      "Number of processed documents: 3061\n",
      "Number of processed documents: 3062\n",
      "Number of processed documents: 3063\n",
      "Number of processed documents: 3064\n",
      "Number of processed documents: 3065\n",
      "Number of processed documents: 3066\n",
      "Number of processed documents: 3067\n",
      "Number of processed documents: 3068\n",
      "Number of processed documents: 3069\n",
      "Number of processed documents: 3070\n",
      "Number of processed documents: 3071\n",
      "Number of processed documents: 3072\n",
      "Number of processed documents: 3073\n",
      "Number of processed documents: 3074\n",
      "Number of processed documents: 3075\n",
      "Number of processed documents: 3076\n",
      "Number of processed documents: 3077\n",
      "Number of processed documents: 3078\n",
      "Number of processed documents: 3079\n",
      "Number of processed documents: 3080\n",
      "Number of processed documents: 3081\n",
      "Number of processed documents: 3082\n",
      "Number of processed documents: 3083\n",
      "Number of processed documents: 3084\n",
      "Number of processed documents: 3085\n",
      "Number of processed documents: 3086\n",
      "Number of processed documents: 3087\n",
      "Number of processed documents: 3088\n",
      "Number of processed documents: 3089\n",
      "Number of processed documents: 3090\n",
      "Number of processed documents: 3091\n",
      "Number of processed documents: 3092\n",
      "Number of processed documents: 3093\n",
      "Number of processed documents: 3094\n",
      "Number of processed documents: 3095\n",
      "Number of processed documents: 3096\n",
      "Number of processed documents: 3097\n",
      "Number of processed documents: 3098\n",
      "Number of processed documents: 3099\n",
      "Number of processed documents: 3100\n",
      "Number of processed documents: 3101\n",
      "Number of processed documents: 3102\n",
      "Number of processed documents: 3103\n",
      "Number of processed documents: 3104\n",
      "Number of processed documents: 3105\n",
      "Number of processed documents: 3106\n",
      "Number of processed documents: 3107\n",
      "Number of processed documents: 3108\n",
      "Number of processed documents: 3109\n",
      "Number of processed documents: 3110\n",
      "Number of processed documents: 3111\n",
      "Number of processed documents: 3112\n",
      "Number of processed documents: 3113\n",
      "Number of processed documents: 3114\n",
      "Number of processed documents: 3115\n",
      "Number of processed documents: 3116\n",
      "Number of processed documents: 3117\n",
      "Number of processed documents: 3118\n",
      "Number of processed documents: 3119\n",
      "Number of processed documents: 3120\n",
      "Number of processed documents: 3121\n",
      "Number of processed documents: 3122\n",
      "Number of processed documents: 3123\n",
      "Number of processed documents: 3124\n",
      "Number of processed documents: 3125\n",
      "Number of processed documents: 3126\n",
      "Number of processed documents: 3127\n",
      "Number of processed documents: 3128\n",
      "Number of processed documents: 3129\n",
      "Number of processed documents: 3130\n",
      "Number of processed documents: 3131\n",
      "Number of processed documents: 3132\n",
      "Number of processed documents: 3133\n",
      "Number of processed documents: 3134\n",
      "Number of processed documents: 3135\n",
      "Number of processed documents: 3136\n",
      "Number of processed documents: 3137\n",
      "Number of processed documents: 3138\n",
      "Number of processed documents: 3139\n",
      "Number of processed documents: 3140\n",
      "Number of processed documents: 3141\n",
      "Number of processed documents: 3142\n",
      "Number of processed documents: 3143\n",
      "Number of processed documents: 3144\n",
      "Number of processed documents: 3145\n",
      "Number of processed documents: 3146\n",
      "Number of processed documents: 3147\n",
      "Number of processed documents: 3148\n",
      "Number of processed documents: 3149\n",
      "Number of processed documents: 3150\n",
      "Number of processed documents: 3151\n",
      "Number of processed documents: 3152\n",
      "Number of processed documents: 3153\n",
      "Number of processed documents: 3154\n",
      "Number of processed documents: 3155\n",
      "Number of processed documents: 3156\n",
      "Number of processed documents: 3157\n",
      "Number of processed documents: 3158\n",
      "Number of processed documents: 3159\n",
      "Number of processed documents: 3160\n",
      "Number of processed documents: 3161\n",
      "Number of processed documents: 3162\n",
      "Number of processed documents: 3163\n",
      "Number of processed documents: 3164\n",
      "Number of processed documents: 3165\n",
      "Number of processed documents: 3166\n",
      "Number of processed documents: 3167\n",
      "Number of processed documents: 3168\n",
      "Number of processed documents: 3169\n",
      "Number of processed documents: 3170\n",
      "Number of processed documents: 3171\n",
      "Number of processed documents: 3172\n",
      "Number of processed documents: 3173\n",
      "Number of processed documents: 3174\n",
      "Number of processed documents: 3175\n",
      "Number of processed documents: 3176\n",
      "Number of processed documents: 3177\n",
      "Number of processed documents: 3178\n",
      "Number of processed documents: 3179\n",
      "Number of processed documents: 3180\n",
      "Number of processed documents: 3181\n",
      "Number of processed documents: 3182\n",
      "Number of processed documents: 3183\n",
      "Number of processed documents: 3184\n",
      "Number of processed documents: 3185\n",
      "Number of processed documents: 3186\n",
      "Number of processed documents: 3187\n",
      "Number of processed documents: 3188\n",
      "Number of processed documents: 3189\n",
      "Number of processed documents: 3190\n",
      "Number of processed documents: 3191\n",
      "Number of processed documents: 3192\n",
      "Number of processed documents: 3193\n",
      "Number of processed documents: 3194\n",
      "Number of processed documents: 3195\n",
      "Number of processed documents: 3196\n",
      "Number of processed documents: 3197\n",
      "Number of processed documents: 3198\n",
      "Number of processed documents: 3199\n",
      "Number of processed documents: 3200\n",
      "Number of processed documents: 3201\n",
      "Number of processed documents: 3202\n",
      "Number of processed documents: 3203\n",
      "Number of processed documents: 3204\n",
      "Number of processed documents: 3205\n",
      "Number of processed documents: 3206\n",
      "Number of processed documents: 3207\n",
      "Number of processed documents: 3208\n",
      "Number of processed documents: 3209\n",
      "Number of processed documents: 3210\n",
      "Number of processed documents: 3211\n",
      "Number of processed documents: 3212\n",
      "Number of processed documents: 3213\n",
      "Number of processed documents: 3214\n",
      "Number of processed documents: 3215\n",
      "Number of processed documents: 3216\n",
      "Number of processed documents: 3217\n",
      "Number of processed documents: 3218\n",
      "Number of processed documents: 3219\n",
      "Number of processed documents: 3220\n",
      "Number of processed documents: 3221\n",
      "Number of processed documents: 3222\n",
      "Number of processed documents: 3223\n",
      "Number of processed documents: 3224\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 3225\n",
      "Number of processed documents: 3226\n",
      "Number of processed documents: 3227\n",
      "Number of processed documents: 3228\n",
      "Number of processed documents: 3229\n",
      "Number of processed documents: 3230\n",
      "Number of processed documents: 3231\n",
      "Number of processed documents: 3232\n",
      "Number of processed documents: 3233\n",
      "Number of processed documents: 3234\n",
      "Number of processed documents: 3235\n",
      "Number of processed documents: 3236\n",
      "Number of processed documents: 3237\n",
      "Number of processed documents: 3238\n",
      "Number of processed documents: 3239\n",
      "Number of processed documents: 3240\n",
      "Number of processed documents: 3241\n",
      "Number of processed documents: 3242\n",
      "Number of processed documents: 3243\n",
      "Number of processed documents: 3244\n",
      "Number of processed documents: 3245\n",
      "Number of processed documents: 3246\n",
      "Number of processed documents: 3247\n",
      "Number of processed documents: 3248\n",
      "Number of processed documents: 3249\n",
      "Number of processed documents: 3250\n",
      "Number of processed documents: 3251\n",
      "Number of processed documents: 3252\n",
      "Number of processed documents: 3253\n",
      "Number of processed documents: 3254\n",
      "Number of processed documents: 3255\n",
      "Number of processed documents: 3256\n",
      "Number of processed documents: 3257\n",
      "Number of processed documents: 3258\n",
      "Number of processed documents: 3259\n",
      "Number of processed documents: 3260\n",
      "Number of processed documents: 3261\n",
      "Number of processed documents: 3262\n",
      "Number of processed documents: 3263\n",
      "Number of processed documents: 3264\n",
      "Number of processed documents: 3265\n",
      "Number of processed documents: 3266\n",
      "Number of processed documents: 3267\n",
      "Number of processed documents: 3268\n",
      "Number of processed documents: 3269\n",
      "Number of processed documents: 3270\n",
      "Number of processed documents: 3271\n",
      "Number of processed documents: 3272\n",
      "Number of processed documents: 3273\n",
      "Number of processed documents: 3274\n",
      "Number of processed documents: 3275\n",
      "Number of processed documents: 3276\n",
      "Number of processed documents: 3277\n",
      "Number of processed documents: 3278\n",
      "Number of processed documents: 3279\n",
      "Number of processed documents: 3280\n",
      "Number of processed documents: 3281\n",
      "Number of processed documents: 3282\n",
      "Number of processed documents: 3283\n",
      "Number of processed documents: 3284\n",
      "Number of processed documents: 3285\n",
      "Number of processed documents: 3286\n",
      "Number of processed documents: 3287\n",
      "Number of processed documents: 3288\n",
      "Number of processed documents: 3289\n",
      "Number of processed documents: 3290\n",
      "Number of processed documents: 3291\n",
      "Number of processed documents: 3292\n",
      "Number of processed documents: 3293\n",
      "Number of processed documents: 3294\n",
      "Number of processed documents: 3295\n",
      "Number of processed documents: 3296\n",
      "Number of processed documents: 3297\n",
      "Number of processed documents: 3298\n",
      "Number of processed documents: 3299\n",
      "Number of processed documents: 3300\n",
      "Number of processed documents: 3301\n",
      "Number of processed documents: 3302\n",
      "Number of processed documents: 3303\n",
      "Number of processed documents: 3304\n",
      "Number of processed documents: 3305\n",
      "Number of processed documents: 3306\n",
      "Number of processed documents: 3307\n",
      "Number of processed documents: 3308\n",
      "Number of processed documents: 3309\n",
      "Number of processed documents: 3310\n",
      "Number of processed documents: 3311\n",
      "Number of processed documents: 3312\n",
      "Number of processed documents: 3313\n",
      "Number of processed documents: 3314\n",
      "Number of processed documents: 3315\n",
      "Number of processed documents: 3316\n",
      "Number of processed documents: 3317\n",
      "Number of processed documents: 3318\n",
      "Number of processed documents: 3319\n",
      "Number of processed documents: 3320\n",
      "Number of processed documents: 3321\n",
      "Number of processed documents: 3322\n",
      "Number of processed documents: 3323\n",
      "Number of processed documents: 3324\n",
      "Number of processed documents: 3325\n",
      "Number of processed documents: 3326\n",
      "Number of processed documents: 3327\n",
      "Number of processed documents: 3328\n",
      "Number of processed documents: 3329\n",
      "Number of processed documents: 3330\n",
      "Number of processed documents: 3331\n",
      "Number of processed documents: 3332\n",
      "Number of processed documents: 3333\n",
      "Number of processed documents: 3334\n",
      "Number of processed documents: 3335\n",
      "Number of processed documents: 3336\n",
      "Number of processed documents: 3337\n",
      "Number of processed documents: 3338\n",
      "Number of processed documents: 3339\n",
      "Number of processed documents: 3340\n",
      "Number of processed documents: 3341\n",
      "Number of processed documents: 3342\n",
      "Number of processed documents: 3343\n",
      "Number of processed documents: 3344\n",
      "Number of processed documents: 3345\n",
      "Number of processed documents: 3346\n",
      "Number of processed documents: 3347\n",
      "Number of processed documents: 3348\n",
      "Number of processed documents: 3349\n",
      "Number of processed documents: 3350\n",
      "Number of processed documents: 3351\n",
      "Number of processed documents: 3352\n",
      "Number of processed documents: 3353\n",
      "Number of processed documents: 3354\n",
      "Number of processed documents: 3355\n",
      "Number of processed documents: 3356\n",
      "Number of processed documents: 3357\n",
      "Number of processed documents: 3358\n",
      "Number of processed documents: 3359\n",
      "Number of processed documents: 3360\n",
      "Number of processed documents: 3361\n",
      "Number of processed documents: 3362\n",
      "Number of processed documents: 3363\n",
      "Number of processed documents: 3364\n",
      "Number of processed documents: 3365\n",
      "Number of processed documents: 3366\n",
      "Number of processed documents: 3367\n",
      "Number of processed documents: 3368\n",
      "Number of processed documents: 3369\n",
      "Number of processed documents: 3370\n",
      "Number of processed documents: 3371\n",
      "Number of processed documents: 3372\n",
      "Number of processed documents: 3373\n",
      "Number of processed documents: 3374\n",
      "Number of processed documents: 3375\n",
      "Number of processed documents: 3376\n",
      "Number of processed documents: 3377\n",
      "Number of processed documents: 3378\n",
      "Number of processed documents: 3379\n",
      "Number of processed documents: 3380\n",
      "Number of processed documents: 3381\n",
      "Number of processed documents: 3382\n",
      "Number of processed documents: 3383\n",
      "Number of processed documents: 3384\n",
      "Number of processed documents: 3385\n",
      "Number of processed documents: 3386\n",
      "Number of processed documents: 3387\n",
      "Number of processed documents: 3388\n",
      "Number of processed documents: 3389\n",
      "Number of processed documents: 3390\n",
      "Number of processed documents: 3391\n",
      "Number of processed documents: 3392\n",
      "Number of processed documents: 3393\n",
      "Number of processed documents: 3394\n",
      "Number of processed documents: 3395\n",
      "Number of processed documents: 3396\n",
      "Number of processed documents: 3397\n",
      "Number of processed documents: 3398\n",
      "Number of processed documents: 3399\n",
      "Number of processed documents: 3400\n",
      "Number of processed documents: 3401\n",
      "Number of processed documents: 3402\n",
      "Number of processed documents: 3403\n",
      "Number of processed documents: 3404\n",
      "Number of processed documents: 3405\n",
      "Number of processed documents: 3406\n",
      "Number of processed documents: 3407\n",
      "Number of processed documents: 3408\n",
      "Number of processed documents: 3409\n",
      "Number of processed documents: 3410\n",
      "Number of processed documents: 3411\n",
      "Number of processed documents: 3412\n",
      "Number of processed documents: 3413\n",
      "Number of processed documents: 3414\n",
      "Number of processed documents: 3415\n",
      "Number of processed documents: 3416\n",
      "Number of processed documents: 3417\n",
      "Number of processed documents: 3418\n",
      "Number of processed documents: 3419\n",
      "Number of processed documents: 3420\n",
      "Number of processed documents: 3421\n",
      "Number of processed documents: 3422\n",
      "Number of processed documents: 3423\n",
      "Number of processed documents: 3424\n",
      "Number of processed documents: 3425\n",
      "Number of processed documents: 3426\n",
      "Number of processed documents: 3427\n",
      "Number of processed documents: 3428\n",
      "Number of processed documents: 3429\n",
      "Number of processed documents: 3430\n",
      "Number of processed documents: 3431\n",
      "Number of processed documents: 3432\n",
      "Number of processed documents: 3433\n",
      "Number of processed documents: 3434\n",
      "Number of processed documents: 3435\n",
      "Number of processed documents: 3436\n",
      "Number of processed documents: 3437\n",
      "Number of processed documents: 3438\n",
      "Number of processed documents: 3439\n",
      "Number of processed documents: 3440\n",
      "Number of processed documents: 3441\n",
      "Number of processed documents: 3442\n",
      "Number of processed documents: 3443\n",
      "Number of processed documents: 3444\n",
      "Number of processed documents: 3445\n",
      "Number of processed documents: 3446\n",
      "Number of processed documents: 3447\n",
      "Number of processed documents: 3448\n",
      "Number of processed documents: 3449\n",
      "Number of processed documents: 3450\n",
      "Number of processed documents: 3451\n",
      "Number of processed documents: 3452\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 3453\n",
      "Number of processed documents: 3454\n",
      "Number of processed documents: 3455\n",
      "Number of processed documents: 3456\n",
      "Number of processed documents: 3457\n",
      "Number of processed documents: 3458\n",
      "Number of processed documents: 3459\n",
      "Number of processed documents: 3460\n",
      "Number of processed documents: 3461\n",
      "Number of processed documents: 3462\n",
      "Number of processed documents: 3463\n",
      "Number of processed documents: 3464\n",
      "Number of processed documents: 3465\n",
      "Number of processed documents: 3466\n",
      "Number of processed documents: 3467\n",
      "Number of processed documents: 3468\n",
      "Number of processed documents: 3469\n",
      "Number of processed documents: 3470\n",
      "Number of processed documents: 3471\n",
      "Number of processed documents: 3472\n",
      "Number of processed documents: 3473\n",
      "Number of processed documents: 3474\n",
      "Number of processed documents: 3475\n",
      "Number of processed documents: 3476\n",
      "Number of processed documents: 3477\n",
      "Number of processed documents: 3478\n",
      "Number of processed documents: 3479\n",
      "Number of processed documents: 3480\n",
      "Number of processed documents: 3481\n",
      "Number of processed documents: 3482\n",
      "Number of processed documents: 3483\n",
      "Number of processed documents: 3484\n",
      "Number of processed documents: 3485\n",
      "Number of processed documents: 3486\n",
      "Number of processed documents: 3487\n",
      "Number of processed documents: 3488\n",
      "Number of processed documents: 3489\n",
      "Number of processed documents: 3490\n",
      "Number of processed documents: 3491\n",
      "Number of processed documents: 3492\n",
      "Number of processed documents: 3493\n",
      "Number of processed documents: 3494\n",
      "Number of processed documents: 3495\n",
      "Number of processed documents: 3496\n",
      "Number of processed documents: 3497\n",
      "Number of processed documents: 3498\n",
      "Number of processed documents: 3499\n",
      "Number of processed documents: 3500\n",
      "Number of processed documents: 3501\n",
      "Number of processed documents: 3502\n",
      "Number of processed documents: 3503\n",
      "Number of processed documents: 3504\n",
      "Number of processed documents: 3505\n",
      "Number of processed documents: 3506\n",
      "Number of processed documents: 3507\n",
      "Number of processed documents: 3508\n",
      "Number of processed documents: 3509\n",
      "Number of processed documents: 3510\n",
      "Number of processed documents: 3511\n",
      "Number of processed documents: 3512\n",
      "Number of processed documents: 3513\n",
      "Number of processed documents: 3514\n",
      "Number of processed documents: 3515\n",
      "Number of processed documents: 3516\n",
      "Number of processed documents: 3517\n",
      "Number of processed documents: 3518\n",
      "Number of processed documents: 3519\n",
      "Number of processed documents: 3520\n",
      "Number of processed documents: 3521\n",
      "Number of processed documents: 3522\n",
      "Number of processed documents: 3523\n",
      "Number of processed documents: 3524\n",
      "Number of processed documents: 3525\n",
      "Number of processed documents: 3526\n",
      "Number of processed documents: 3527\n",
      "Number of processed documents: 3528\n",
      "Number of processed documents: 3529\n",
      "Number of processed documents: 3530\n",
      "Number of processed documents: 3531\n",
      "Number of processed documents: 3532\n",
      "Number of processed documents: 3533\n",
      "Number of processed documents: 3534\n",
      "Number of processed documents: 3535\n",
      "Number of processed documents: 3536\n",
      "Number of processed documents: 3537\n",
      "Number of processed documents: 3538\n",
      "Number of processed documents: 3539\n",
      "Number of processed documents: 3540\n",
      "Number of processed documents: 3541\n",
      "Number of processed documents: 3542\n",
      "Number of processed documents: 3543\n",
      "Number of processed documents: 3544\n",
      "Number of processed documents: 3545\n",
      "Number of processed documents: 3546\n",
      "Number of processed documents: 3547\n",
      "Number of processed documents: 3548\n",
      "Number of processed documents: 3549\n",
      "Number of processed documents: 3550\n",
      "Number of processed documents: 3551\n",
      "Number of processed documents: 3552\n",
      "Number of processed documents: 3553\n",
      "Number of processed documents: 3554\n",
      "Number of processed documents: 3555\n",
      "Number of processed documents: 3556\n",
      "Number of processed documents: 3557\n",
      "Number of processed documents: 3558\n",
      "Number of processed documents: 3559\n",
      "Number of processed documents: 3560\n",
      "Number of processed documents: 3561\n",
      "Number of processed documents: 3562\n",
      "Number of processed documents: 3563\n",
      "Number of processed documents: 3564\n",
      "Number of processed documents: 3565\n",
      "Number of processed documents: 3566\n",
      "Number of processed documents: 3567\n",
      "Number of processed documents: 3568\n",
      "Number of processed documents: 3569\n",
      "Number of processed documents: 3570\n",
      "Number of processed documents: 3571\n",
      "Number of processed documents: 3572\n",
      "Number of processed documents: 3573\n",
      "Number of processed documents: 3574\n",
      "Number of processed documents: 3575\n",
      "Number of processed documents: 3576\n",
      "Number of processed documents: 3577\n",
      "Number of processed documents: 3578\n",
      "Number of processed documents: 3579\n",
      "Number of processed documents: 3580\n",
      "Number of processed documents: 3581\n",
      "Number of processed documents: 3582\n",
      "Number of processed documents: 3583\n",
      "Number of processed documents: 3584\n",
      "Number of processed documents: 3585\n",
      "Number of processed documents: 3586\n",
      "Number of processed documents: 3587\n",
      "Number of processed documents: 3588\n",
      "Number of processed documents: 3589\n",
      "Number of processed documents: 3590\n",
      "Number of processed documents: 3591\n",
      "Number of processed documents: 3592\n",
      "Number of processed documents: 3593\n",
      "Number of processed documents: 3594\n",
      "Number of processed documents: 3595\n",
      "Number of processed documents: 3596\n",
      "Number of processed documents: 3597\n",
      "Number of processed documents: 3598\n",
      "Number of processed documents: 3599\n",
      "Number of processed documents: 3600\n",
      "Number of processed documents: 3601\n",
      "Number of processed documents: 3602\n",
      "Number of processed documents: 3603\n",
      "Number of processed documents: 3604\n",
      "Number of processed documents: 3605\n",
      "Number of processed documents: 3606\n",
      "Number of processed documents: 3607\n",
      "Number of processed documents: 3608\n",
      "Number of processed documents: 3609\n",
      "Number of processed documents: 3610\n",
      "Number of processed documents: 3611\n",
      "Number of processed documents: 3612\n",
      "Number of processed documents: 3613\n",
      "Number of processed documents: 3614\n",
      "Number of processed documents: 3615\n",
      "Number of processed documents: 3616\n",
      "Number of processed documents: 3617\n",
      "Number of processed documents: 3618\n",
      "Number of processed documents: 3619\n",
      "Number of processed documents: 3620\n",
      "Number of processed documents: 3621\n",
      "Number of processed documents: 3622\n",
      "Number of processed documents: 3623\n",
      "Number of processed documents: 3624\n",
      "Number of processed documents: 3625\n",
      "Number of processed documents: 3626\n",
      "Number of processed documents: 3627\n",
      "Number of processed documents: 3628\n",
      "Number of processed documents: 3629\n",
      "Number of processed documents: 3630\n",
      "Number of processed documents: 3631\n",
      "Number of processed documents: 3632\n",
      "Number of processed documents: 3633\n",
      "Number of processed documents: 3634\n",
      "Number of processed documents: 3635\n",
      "Number of processed documents: 3636\n",
      "Number of processed documents: 3637\n",
      "Number of processed documents: 3638\n",
      "Number of processed documents: 3639\n",
      "Number of processed documents: 3640\n",
      "Number of processed documents: 3641\n",
      "Number of processed documents: 3642\n",
      "Number of processed documents: 3643\n",
      "Number of processed documents: 3644\n",
      "Number of processed documents: 3645\n",
      "Number of processed documents: 3646\n",
      "Number of processed documents: 3647\n",
      "Number of processed documents: 3648\n",
      "Number of processed documents: 3649\n",
      "Number of processed documents: 3650\n",
      "Number of processed documents: 3651\n",
      "Number of processed documents: 3652\n",
      "Number of processed documents: 3653\n",
      "Number of processed documents: 3654\n",
      "Number of processed documents: 3655\n",
      "Number of processed documents: 3656\n",
      "Number of processed documents: 3657\n",
      "Number of processed documents: 3658\n",
      "Number of processed documents: 3659\n",
      "Number of processed documents: 3660\n",
      "Number of processed documents: 3661\n",
      "Number of processed documents: 3662\n",
      "Number of processed documents: 3663\n",
      "Number of processed documents: 3664\n",
      "Number of processed documents: 3665\n",
      "Number of processed documents: 3666\n",
      "Number of processed documents: 3667\n",
      "Number of processed documents: 3668\n",
      "Number of processed documents: 3669\n",
      "Number of processed documents: 3670\n",
      "Number of processed documents: 3671\n",
      "Number of processed documents: 3672\n",
      "Number of processed documents: 3673\n",
      "Number of processed documents: 3674\n",
      "Number of processed documents: 3675\n",
      "Number of processed documents: 3676\n",
      "Number of processed documents: 3677\n",
      "Number of processed documents: 3678\n",
      "Number of processed documents: 3679\n",
      "Number of processed documents: 3680\n",
      "Number of processed documents: 3681\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 3682\n",
      "Number of processed documents: 3683\n",
      "Number of processed documents: 3684\n",
      "Number of processed documents: 3685\n",
      "Number of processed documents: 3686\n",
      "Number of processed documents: 3687\n",
      "Number of processed documents: 3688\n",
      "Number of processed documents: 3689\n",
      "Number of processed documents: 3690\n",
      "Number of processed documents: 3691\n",
      "Number of processed documents: 3692\n",
      "Number of processed documents: 3693\n",
      "Number of processed documents: 3694\n",
      "Number of processed documents: 3695\n",
      "Number of processed documents: 3696\n",
      "Number of processed documents: 3697\n",
      "Number of processed documents: 3698\n",
      "Number of processed documents: 3699\n",
      "Number of processed documents: 3700\n",
      "Number of processed documents: 3701\n",
      "Number of processed documents: 3702\n",
      "Number of processed documents: 3703\n",
      "Number of processed documents: 3704\n",
      "Number of processed documents: 3705\n",
      "Number of processed documents: 3706\n",
      "Number of processed documents: 3707\n",
      "Number of processed documents: 3708\n",
      "Number of processed documents: 3709\n",
      "Number of processed documents: 3710\n",
      "Number of processed documents: 3711\n",
      "Number of processed documents: 3712\n",
      "Number of processed documents: 3713\n",
      "Number of processed documents: 3714\n",
      "Number of processed documents: 3715\n",
      "Number of processed documents: 3716\n",
      "Number of processed documents: 3717\n",
      "Number of processed documents: 3718\n",
      "Number of processed documents: 3719\n",
      "Number of processed documents: 3720\n",
      "Number of processed documents: 3721\n",
      "Number of processed documents: 3722\n",
      "Number of processed documents: 3723\n",
      "Number of processed documents: 3724\n",
      "Number of processed documents: 3725\n",
      "Number of processed documents: 3726\n",
      "Number of processed documents: 3727\n",
      "Number of processed documents: 3728\n",
      "Number of processed documents: 3729\n",
      "Number of processed documents: 3730\n",
      "Number of processed documents: 3731\n",
      "Number of processed documents: 3732\n",
      "Number of processed documents: 3733\n",
      "Number of processed documents: 3734\n",
      "Number of processed documents: 3735\n",
      "Number of processed documents: 3736\n",
      "Number of processed documents: 3737\n",
      "Number of processed documents: 3738\n",
      "Number of processed documents: 3739\n",
      "Number of processed documents: 3740\n",
      "Number of processed documents: 3741\n",
      "Number of processed documents: 3742\n",
      "Number of processed documents: 3743\n",
      "Number of processed documents: 3744\n",
      "Number of processed documents: 3745\n",
      "Number of processed documents: 3746\n",
      "Number of processed documents: 3747\n",
      "Number of processed documents: 3748\n",
      "Number of processed documents: 3749\n",
      "Number of processed documents: 3750\n",
      "Number of processed documents: 3751\n",
      "Number of processed documents: 3752\n",
      "Number of processed documents: 3753\n",
      "Number of processed documents: 3754\n",
      "Number of processed documents: 3755\n",
      "Number of processed documents: 3756\n",
      "Number of processed documents: 3757\n",
      "Number of processed documents: 3758\n",
      "Number of processed documents: 3759\n",
      "Number of processed documents: 3760\n",
      "Number of processed documents: 3761\n",
      "Number of processed documents: 3762\n",
      "Number of processed documents: 3763\n",
      "Number of processed documents: 3764\n",
      "Number of processed documents: 3765\n",
      "Number of processed documents: 3766\n",
      "Number of processed documents: 3767\n",
      "Number of processed documents: 3768\n",
      "Number of processed documents: 3769\n",
      "Number of processed documents: 3770\n",
      "Number of processed documents: 3771\n",
      "Number of processed documents: 3772\n",
      "Number of processed documents: 3773\n",
      "Number of processed documents: 3774\n",
      "Number of processed documents: 3775\n",
      "Number of processed documents: 3776\n",
      "Number of processed documents: 3777\n",
      "Number of processed documents: 3778\n",
      "Number of processed documents: 3779\n",
      "Number of processed documents: 3780\n",
      "Number of processed documents: 3781\n",
      "Number of processed documents: 3782\n",
      "Number of processed documents: 3783\n",
      "Number of processed documents: 3784\n",
      "Number of processed documents: 3785\n",
      "Number of processed documents: 3786\n",
      "Number of processed documents: 3787\n",
      "Number of processed documents: 3788\n",
      "Number of processed documents: 3789\n",
      "Number of processed documents: 3790\n",
      "Number of processed documents: 3791\n",
      "Number of processed documents: 3792\n",
      "Number of processed documents: 3793\n",
      "Number of processed documents: 3794\n",
      "Number of processed documents: 3795\n",
      "Number of processed documents: 3796\n",
      "Number of processed documents: 3797\n",
      "Number of processed documents: 3798\n",
      "Number of processed documents: 3799\n",
      "Number of processed documents: 3800\n",
      "Number of processed documents: 3801\n",
      "Number of processed documents: 3802\n",
      "Number of processed documents: 3803\n",
      "Number of processed documents: 3804\n",
      "Number of processed documents: 3805\n",
      "Number of processed documents: 3806\n",
      "Number of processed documents: 3807\n",
      "Number of processed documents: 3808\n",
      "Number of processed documents: 3809\n",
      "Number of processed documents: 3810\n",
      "Number of processed documents: 3811\n",
      "Number of processed documents: 3812\n",
      "Number of processed documents: 3813\n",
      "Number of processed documents: 3814\n",
      "Number of processed documents: 3815\n",
      "Number of processed documents: 3816\n",
      "Number of processed documents: 3817\n",
      "Number of processed documents: 3818\n",
      "Number of processed documents: 3819\n",
      "Number of processed documents: 3820\n",
      "Number of processed documents: 3821\n",
      "Number of processed documents: 3822\n",
      "Number of processed documents: 3823\n",
      "Number of processed documents: 3824\n",
      "Number of processed documents: 3825\n",
      "Number of processed documents: 3826\n",
      "Number of processed documents: 3827\n",
      "Number of processed documents: 3828\n",
      "Number of processed documents: 3829\n",
      "Number of processed documents: 3830\n",
      "Number of processed documents: 3831\n",
      "Number of processed documents: 3832\n",
      "Number of processed documents: 3833\n",
      "Number of processed documents: 3834\n",
      "Number of processed documents: 3835\n",
      "Number of processed documents: 3836\n",
      "Number of processed documents: 3837\n",
      "Number of processed documents: 3838\n",
      "Number of processed documents: 3839\n",
      "Number of processed documents: 3840\n",
      "Number of processed documents: 3841\n",
      "Number of processed documents: 3842\n",
      "Number of processed documents: 3843\n",
      "Number of processed documents: 3844\n",
      "Number of processed documents: 3845\n",
      "Number of processed documents: 3846\n",
      "Number of processed documents: 3847\n",
      "Number of processed documents: 3848\n",
      "Number of processed documents: 3849\n",
      "Number of processed documents: 3850\n",
      "Number of processed documents: 3851\n",
      "Number of processed documents: 3852\n",
      "Number of processed documents: 3853\n",
      "Number of processed documents: 3854\n",
      "Number of processed documents: 3855\n",
      "Number of processed documents: 3856\n",
      "Number of processed documents: 3857\n",
      "Number of processed documents: 3858\n",
      "Number of processed documents: 3859\n",
      "Number of processed documents: 3860\n",
      "Number of processed documents: 3861\n",
      "Number of processed documents: 3862\n",
      "Number of processed documents: 3863\n",
      "Number of processed documents: 3864\n",
      "Number of processed documents: 3865\n",
      "Number of processed documents: 3866\n",
      "Number of processed documents: 3867\n",
      "Number of processed documents: 3868\n",
      "Number of processed documents: 3869\n",
      "Number of processed documents: 3870\n",
      "Number of processed documents: 3871\n",
      "Number of processed documents: 3872\n",
      "Number of processed documents: 3873\n",
      "Number of processed documents: 3874\n",
      "Number of processed documents: 3875\n",
      "Number of processed documents: 3876\n",
      "Number of processed documents: 3877\n",
      "Number of processed documents: 3878\n",
      "Number of processed documents: 3879\n",
      "Number of processed documents: 3880\n",
      "Number of processed documents: 3881\n",
      "Number of processed documents: 3882\n",
      "Number of processed documents: 3883\n",
      "Number of processed documents: 3884\n",
      "Number of processed documents: 3885\n",
      "Number of processed documents: 3886\n",
      "Number of processed documents: 3887\n",
      "Number of processed documents: 3888\n",
      "Number of processed documents: 3889\n",
      "Number of processed documents: 3890\n",
      "Number of processed documents: 3891\n",
      "Number of processed documents: 3892\n",
      "Number of processed documents: 3893\n",
      "Number of processed documents: 3894\n",
      "Number of processed documents: 3895\n",
      "Number of processed documents: 3896\n",
      "Number of processed documents: 3897\n",
      "Number of processed documents: 3898\n",
      "Number of processed documents: 3899\n",
      "Number of processed documents: 3900\n",
      "Number of processed documents: 3901\n",
      "Number of processed documents: 3902\n",
      "Number of processed documents: 3903\n",
      "Number of processed documents: 3904\n",
      "Number of processed documents: 3905\n",
      "Number of processed documents: 3906\n",
      "Number of processed documents: 3907\n",
      "Number of processed documents: 3908\n",
      "Number of processed documents: 3909\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 3910\n",
      "Number of processed documents: 3911\n",
      "Number of processed documents: 3912\n",
      "Number of processed documents: 3913\n",
      "Number of processed documents: 3914\n",
      "Number of processed documents: 3915\n",
      "Number of processed documents: 3916\n",
      "Number of processed documents: 3917\n",
      "Number of processed documents: 3918\n",
      "Number of processed documents: 3919\n",
      "Number of processed documents: 3920\n",
      "Number of processed documents: 3921\n",
      "Number of processed documents: 3922\n",
      "Number of processed documents: 3923\n",
      "Number of processed documents: 3924\n",
      "Number of processed documents: 3925\n",
      "Number of processed documents: 3926\n",
      "Number of processed documents: 3927\n",
      "Number of processed documents: 3928\n",
      "Number of processed documents: 3929\n",
      "Number of processed documents: 3930\n",
      "Number of processed documents: 3931\n",
      "Number of processed documents: 3932\n",
      "Number of processed documents: 3933\n",
      "Number of processed documents: 3934\n",
      "Number of processed documents: 3935\n",
      "Number of processed documents: 3936\n",
      "Number of processed documents: 3937\n",
      "Number of processed documents: 3938\n",
      "Number of processed documents: 3939\n",
      "Number of processed documents: 3940\n",
      "Number of processed documents: 3941\n",
      "Number of processed documents: 3942\n",
      "Number of processed documents: 3943\n",
      "Number of processed documents: 3944\n",
      "Number of processed documents: 3945\n",
      "Number of processed documents: 3946\n",
      "Number of processed documents: 3947\n",
      "Number of processed documents: 3948\n",
      "Number of processed documents: 3949\n",
      "Number of processed documents: 3950\n",
      "Number of processed documents: 3951\n",
      "Number of processed documents: 3952\n",
      "Number of processed documents: 3953\n",
      "Number of processed documents: 3954\n",
      "Number of processed documents: 3955\n",
      "Number of processed documents: 3956\n",
      "Number of processed documents: 3957\n",
      "Number of processed documents: 3958\n",
      "Number of processed documents: 3959\n",
      "Number of processed documents: 3960\n",
      "Number of processed documents: 3961\n",
      "Number of processed documents: 3962\n",
      "Number of processed documents: 3963\n",
      "Number of processed documents: 3964\n",
      "Number of processed documents: 3965\n",
      "Number of processed documents: 3966\n",
      "Number of processed documents: 3967\n",
      "Number of processed documents: 3968\n",
      "Number of processed documents: 3969\n",
      "Number of processed documents: 3970\n",
      "Number of processed documents: 3971\n",
      "Number of processed documents: 3972\n",
      "Number of processed documents: 3973\n",
      "Number of processed documents: 3974\n",
      "Number of processed documents: 3975\n",
      "Number of processed documents: 3976\n",
      "Number of processed documents: 3977\n",
      "Number of processed documents: 3978\n",
      "Number of processed documents: 3979\n",
      "Number of processed documents: 3980\n",
      "Number of processed documents: 3981\n",
      "Number of processed documents: 3982\n",
      "Number of processed documents: 3983\n",
      "Number of processed documents: 3984\n",
      "Number of processed documents: 3985\n",
      "Number of processed documents: 3986\n",
      "Number of processed documents: 3987\n",
      "Number of processed documents: 3988\n",
      "Number of processed documents: 3989\n",
      "Number of processed documents: 3990\n",
      "Number of processed documents: 3991\n",
      "Number of processed documents: 3992\n",
      "Number of processed documents: 3993\n",
      "Number of processed documents: 3994\n",
      "Number of processed documents: 3995\n",
      "Number of processed documents: 3996\n",
      "Number of processed documents: 3997\n",
      "Number of processed documents: 3998\n",
      "Number of processed documents: 3999\n",
      "Number of processed documents: 4000\n",
      "Number of processed documents: 4001\n",
      "Number of processed documents: 4002\n",
      "Number of processed documents: 4003\n",
      "Number of processed documents: 4004\n",
      "Number of processed documents: 4005\n",
      "Number of processed documents: 4006\n",
      "Number of processed documents: 4007\n",
      "Number of processed documents: 4008\n",
      "Number of processed documents: 4009\n",
      "Number of processed documents: 4010\n",
      "Number of processed documents: 4011\n",
      "Number of processed documents: 4012\n",
      "Number of processed documents: 4013\n",
      "Number of processed documents: 4014\n",
      "Number of processed documents: 4015\n",
      "Number of processed documents: 4016\n",
      "Number of processed documents: 4017\n",
      "Number of processed documents: 4018\n",
      "Number of processed documents: 4019\n",
      "Number of processed documents: 4020\n",
      "Number of processed documents: 4021\n",
      "Number of processed documents: 4022\n",
      "Number of processed documents: 4023\n",
      "Number of processed documents: 4024\n",
      "Number of processed documents: 4025\n",
      "Number of processed documents: 4026\n",
      "Number of processed documents: 4027\n",
      "Number of processed documents: 4028\n",
      "Number of processed documents: 4029\n",
      "Number of processed documents: 4030\n",
      "Number of processed documents: 4031\n",
      "Number of processed documents: 4032\n",
      "Number of processed documents: 4033\n",
      "Number of processed documents: 4034\n",
      "Number of processed documents: 4035\n",
      "Number of processed documents: 4036\n",
      "Number of processed documents: 4037\n",
      "Number of processed documents: 4038\n",
      "Number of processed documents: 4039\n",
      "Number of processed documents: 4040\n",
      "Number of processed documents: 4041\n",
      "Number of processed documents: 4042\n",
      "Number of processed documents: 4043\n",
      "Number of processed documents: 4044\n",
      "Number of processed documents: 4045\n",
      "Number of processed documents: 4046\n",
      "Number of processed documents: 4047\n",
      "Number of processed documents: 4048\n",
      "Number of processed documents: 4049\n",
      "Number of processed documents: 4050\n",
      "Number of processed documents: 4051\n",
      "Number of processed documents: 4052\n",
      "Number of processed documents: 4053\n",
      "Number of processed documents: 4054\n",
      "Number of processed documents: 4055\n",
      "Number of processed documents: 4056\n",
      "Number of processed documents: 4057\n",
      "Number of processed documents: 4058\n",
      "Number of processed documents: 4059\n",
      "Number of processed documents: 4060\n",
      "Number of processed documents: 4061\n",
      "Number of processed documents: 4062\n",
      "Number of processed documents: 4063\n",
      "Number of processed documents: 4064\n",
      "Number of processed documents: 4065\n",
      "Number of processed documents: 4066\n",
      "Number of processed documents: 4067\n",
      "Number of processed documents: 4068\n",
      "Number of processed documents: 4069\n",
      "Number of processed documents: 4070\n",
      "Number of processed documents: 4071\n",
      "Number of processed documents: 4072\n",
      "Number of processed documents: 4073\n",
      "Number of processed documents: 4074\n",
      "Number of processed documents: 4075\n",
      "Number of processed documents: 4076\n",
      "Number of processed documents: 4077\n",
      "Number of processed documents: 4078\n",
      "Number of processed documents: 4079\n",
      "Number of processed documents: 4080\n",
      "Number of processed documents: 4081\n",
      "Number of processed documents: 4082\n",
      "Number of processed documents: 4083\n",
      "Number of processed documents: 4084\n",
      "Number of processed documents: 4085\n",
      "Number of processed documents: 4086\n",
      "Number of processed documents: 4087\n",
      "Number of processed documents: 4088\n",
      "Number of processed documents: 4089\n",
      "Number of processed documents: 4090\n",
      "Number of processed documents: 4091\n",
      "Number of processed documents: 4092\n",
      "Number of processed documents: 4093\n",
      "Number of processed documents: 4094\n",
      "Number of processed documents: 4095\n",
      "Number of processed documents: 4096\n",
      "Number of processed documents: 4097\n",
      "Number of processed documents: 4098\n",
      "Number of processed documents: 4099\n",
      "Number of processed documents: 4100\n",
      "Number of processed documents: 4101\n",
      "Number of processed documents: 4102\n",
      "Number of processed documents: 4103\n",
      "Number of processed documents: 4104\n",
      "Number of processed documents: 4105\n",
      "Number of processed documents: 4106\n",
      "Number of processed documents: 4107\n",
      "Number of processed documents: 4108\n",
      "Number of processed documents: 4109\n",
      "Number of processed documents: 4110\n",
      "Number of processed documents: 4111\n",
      "Number of processed documents: 4112\n",
      "Number of processed documents: 4113\n",
      "Number of processed documents: 4114\n",
      "Number of processed documents: 4115\n",
      "Number of processed documents: 4116\n",
      "Number of processed documents: 4117\n",
      "Number of processed documents: 4118\n",
      "Number of processed documents: 4119\n",
      "Number of processed documents: 4120\n",
      "Number of processed documents: 4121\n",
      "Number of processed documents: 4122\n",
      "Number of processed documents: 4123\n",
      "Number of processed documents: 4124\n",
      "Number of processed documents: 4125\n",
      "Number of processed documents: 4126\n",
      "Number of processed documents: 4127\n",
      "Number of processed documents: 4128\n",
      "Number of processed documents: 4129\n",
      "Number of processed documents: 4130\n",
      "Number of processed documents: 4131\n",
      "Number of processed documents: 4132\n",
      "Number of processed documents: 4133\n",
      "Number of processed documents: 4134\n",
      "Number of processed documents: 4135\n",
      "Number of processed documents: 4136\n",
      "Number of processed documents: 4137\n",
      "Number of processed documents: 4138\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 4139\n",
      "Number of processed documents: 4140\n",
      "Number of processed documents: 4141\n",
      "Number of processed documents: 4142\n",
      "Number of processed documents: 4143\n",
      "Number of processed documents: 4144\n",
      "Number of processed documents: 4145\n",
      "Number of processed documents: 4146\n",
      "Number of processed documents: 4147\n",
      "Number of processed documents: 4148\n",
      "Number of processed documents: 4149\n",
      "Number of processed documents: 4150\n",
      "Number of processed documents: 4151\n",
      "Number of processed documents: 4152\n",
      "Number of processed documents: 4153\n",
      "Number of processed documents: 4154\n",
      "Number of processed documents: 4155\n",
      "Number of processed documents: 4156\n",
      "Number of processed documents: 4157\n",
      "Number of processed documents: 4158\n",
      "Number of processed documents: 4159\n",
      "Number of processed documents: 4160\n",
      "Number of processed documents: 4161\n",
      "Number of processed documents: 4162\n",
      "Number of processed documents: 4163\n",
      "Number of processed documents: 4164\n",
      "Number of processed documents: 4165\n",
      "Number of processed documents: 4166\n",
      "Number of processed documents: 4167\n",
      "Number of processed documents: 4168\n",
      "Number of processed documents: 4169\n",
      "Number of processed documents: 4170\n",
      "Number of processed documents: 4171\n",
      "Number of processed documents: 4172\n",
      "Number of processed documents: 4173\n",
      "Number of processed documents: 4174\n",
      "Number of processed documents: 4175\n",
      "Number of processed documents: 4176\n",
      "Number of processed documents: 4177\n",
      "Number of processed documents: 4178\n",
      "Number of processed documents: 4179\n",
      "Number of processed documents: 4180\n",
      "Number of processed documents: 4181\n",
      "Number of processed documents: 4182\n",
      "Number of processed documents: 4183\n",
      "Number of processed documents: 4184\n",
      "Number of processed documents: 4185\n",
      "Number of processed documents: 4186\n",
      "Number of processed documents: 4187\n",
      "Number of processed documents: 4188\n",
      "Number of processed documents: 4189\n",
      "Number of processed documents: 4190\n",
      "Number of processed documents: 4191\n",
      "Number of processed documents: 4192\n",
      "Number of processed documents: 4193\n",
      "Number of processed documents: 4194\n",
      "Number of processed documents: 4195\n",
      "Number of processed documents: 4196\n",
      "Number of processed documents: 4197\n",
      "Number of processed documents: 4198\n",
      "Number of processed documents: 4199\n",
      "Number of processed documents: 4200\n",
      "Number of processed documents: 4201\n",
      "Number of processed documents: 4202\n",
      "Number of processed documents: 4203\n",
      "Number of processed documents: 4204\n",
      "Number of processed documents: 4205\n",
      "Number of processed documents: 4206\n",
      "Number of processed documents: 4207\n",
      "Number of processed documents: 4208\n",
      "Number of processed documents: 4209\n",
      "Number of processed documents: 4210\n",
      "Number of processed documents: 4211\n",
      "Number of processed documents: 4212\n",
      "Number of processed documents: 4213\n",
      "Number of processed documents: 4214\n",
      "Number of processed documents: 4215\n",
      "Number of processed documents: 4216\n",
      "Number of processed documents: 4217\n",
      "Number of processed documents: 4218\n",
      "Number of processed documents: 4219\n",
      "Number of processed documents: 4220\n",
      "Number of processed documents: 4221\n",
      "Number of processed documents: 4222\n",
      "Number of processed documents: 4223\n",
      "Number of processed documents: 4224\n",
      "Number of processed documents: 4225\n",
      "Number of processed documents: 4226\n",
      "Number of processed documents: 4227\n",
      "Number of processed documents: 4228\n",
      "Number of processed documents: 4229\n",
      "Number of processed documents: 4230\n",
      "Number of processed documents: 4231\n",
      "Number of processed documents: 4232\n",
      "Number of processed documents: 4233\n",
      "Number of processed documents: 4234\n",
      "Number of processed documents: 4235\n",
      "Number of processed documents: 4236\n",
      "Number of processed documents: 4237\n",
      "Number of processed documents: 4238\n",
      "Number of processed documents: 4239\n",
      "Number of processed documents: 4240\n",
      "Number of processed documents: 4241\n",
      "Number of processed documents: 4242\n",
      "Number of processed documents: 4243\n",
      "Number of processed documents: 4244\n",
      "Number of processed documents: 4245\n",
      "Number of processed documents: 4246\n",
      "Number of processed documents: 4247\n",
      "Number of processed documents: 4248\n",
      "Number of processed documents: 4249\n",
      "Number of processed documents: 4250\n",
      "Number of processed documents: 4251\n",
      "Number of processed documents: 4252\n",
      "Number of processed documents: 4253\n",
      "Number of processed documents: 4254\n",
      "Number of processed documents: 4255\n",
      "Number of processed documents: 4256\n",
      "Number of processed documents: 4257\n",
      "Number of processed documents: 4258\n",
      "Number of processed documents: 4259\n",
      "Number of processed documents: 4260\n",
      "Number of processed documents: 4261\n",
      "Number of processed documents: 4262\n",
      "Number of processed documents: 4263\n",
      "Number of processed documents: 4264\n",
      "Number of processed documents: 4265\n",
      "Number of processed documents: 4266\n",
      "Number of processed documents: 4267\n",
      "Number of processed documents: 4268\n",
      "Number of processed documents: 4269\n",
      "Number of processed documents: 4270\n",
      "Number of processed documents: 4271\n",
      "Number of processed documents: 4272\n",
      "Number of processed documents: 4273\n",
      "Number of processed documents: 4274\n",
      "Number of processed documents: 4275\n",
      "Number of processed documents: 4276\n",
      "Number of processed documents: 4277\n",
      "Number of processed documents: 4278\n",
      "Number of processed documents: 4279\n",
      "Number of processed documents: 4280\n",
      "Number of processed documents: 4281\n",
      "Number of processed documents: 4282\n",
      "Number of processed documents: 4283\n",
      "Number of processed documents: 4284\n",
      "Number of processed documents: 4285\n",
      "Number of processed documents: 4286\n",
      "Number of processed documents: 4287\n",
      "Number of processed documents: 4288\n",
      "Number of processed documents: 4289\n",
      "Number of processed documents: 4290\n",
      "Number of processed documents: 4291\n",
      "Number of processed documents: 4292\n",
      "Number of processed documents: 4293\n",
      "Number of processed documents: 4294\n",
      "Number of processed documents: 4295\n",
      "Number of processed documents: 4296\n",
      "Number of processed documents: 4297\n",
      "Number of processed documents: 4298\n",
      "Number of processed documents: 4299\n",
      "Number of processed documents: 4300\n",
      "Number of processed documents: 4301\n",
      "Number of processed documents: 4302\n",
      "Number of processed documents: 4303\n",
      "Number of processed documents: 4304\n",
      "Number of processed documents: 4305\n",
      "Number of processed documents: 4306\n",
      "Number of processed documents: 4307\n",
      "Number of processed documents: 4308\n",
      "Number of processed documents: 4309\n",
      "Number of processed documents: 4310\n",
      "Number of processed documents: 4311\n",
      "Number of processed documents: 4312\n",
      "Number of processed documents: 4313\n",
      "Number of processed documents: 4314\n",
      "Number of processed documents: 4315\n",
      "Number of processed documents: 4316\n",
      "Number of processed documents: 4317\n",
      "Number of processed documents: 4318\n",
      "Number of processed documents: 4319\n",
      "Number of processed documents: 4320\n",
      "Number of processed documents: 4321\n",
      "Number of processed documents: 4322\n",
      "Number of processed documents: 4323\n",
      "Number of processed documents: 4324\n",
      "Number of processed documents: 4325\n",
      "Number of processed documents: 4326\n",
      "Number of processed documents: 4327\n",
      "Number of processed documents: 4328\n",
      "Number of processed documents: 4329\n",
      "Number of processed documents: 4330\n",
      "Number of processed documents: 4331\n",
      "Number of processed documents: 4332\n",
      "Number of processed documents: 4333\n",
      "Number of processed documents: 4334\n",
      "Number of processed documents: 4335\n",
      "Number of processed documents: 4336\n",
      "Number of processed documents: 4337\n",
      "Number of processed documents: 4338\n",
      "Number of processed documents: 4339\n",
      "Number of processed documents: 4340\n",
      "Number of processed documents: 4341\n",
      "Number of processed documents: 4342\n",
      "Number of processed documents: 4343\n",
      "Number of processed documents: 4344\n",
      "Number of processed documents: 4345\n",
      "Number of processed documents: 4346\n",
      "Number of processed documents: 4347\n",
      "Number of processed documents: 4348\n",
      "Number of processed documents: 4349\n",
      "Number of processed documents: 4350\n",
      "Number of processed documents: 4351\n",
      "Number of processed documents: 4352\n",
      "Number of processed documents: 4353\n",
      "Number of processed documents: 4354\n",
      "Number of processed documents: 4355\n",
      "Number of processed documents: 4356\n",
      "Number of processed documents: 4357\n",
      "Number of processed documents: 4358\n",
      "Number of processed documents: 4359\n",
      "Number of processed documents: 4360\n",
      "Number of processed documents: 4361\n",
      "Number of processed documents: 4362\n",
      "Number of processed documents: 4363\n",
      "Number of processed documents: 4364\n",
      "Number of processed documents: 4365\n",
      "Number of processed documents: 4366\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 4367\n",
      "Number of processed documents: 4368\n",
      "Number of processed documents: 4369\n",
      "Number of processed documents: 4370\n",
      "Number of processed documents: 4371\n",
      "Number of processed documents: 4372\n",
      "Number of processed documents: 4373\n",
      "Number of processed documents: 4374\n",
      "Number of processed documents: 4375\n",
      "Number of processed documents: 4376\n",
      "Number of processed documents: 4377\n",
      "Number of processed documents: 4378\n",
      "Number of processed documents: 4379\n",
      "Number of processed documents: 4380\n",
      "Number of processed documents: 4381\n",
      "Number of processed documents: 4382\n",
      "Number of processed documents: 4383\n",
      "Number of processed documents: 4384\n",
      "Number of processed documents: 4385\n",
      "Number of processed documents: 4386\n",
      "Number of processed documents: 4387\n",
      "Number of processed documents: 4388\n",
      "Number of processed documents: 4389\n",
      "Number of processed documents: 4390\n",
      "Number of processed documents: 4391\n",
      "Number of processed documents: 4392\n",
      "Number of processed documents: 4393\n",
      "Number of processed documents: 4394\n",
      "Number of processed documents: 4395\n",
      "Number of processed documents: 4396\n",
      "Number of processed documents: 4397\n",
      "Number of processed documents: 4398\n",
      "Number of processed documents: 4399\n",
      "Number of processed documents: 4400\n",
      "Number of processed documents: 4401\n",
      "Number of processed documents: 4402\n",
      "Number of processed documents: 4403\n",
      "Number of processed documents: 4404\n",
      "Number of processed documents: 4405\n",
      "Number of processed documents: 4406\n",
      "Number of processed documents: 4407\n",
      "Number of processed documents: 4408\n",
      "Number of processed documents: 4409\n",
      "Number of processed documents: 4410\n",
      "Number of processed documents: 4411\n",
      "Number of processed documents: 4412\n",
      "Number of processed documents: 4413\n",
      "Number of processed documents: 4414\n",
      "Number of processed documents: 4415\n",
      "Number of processed documents: 4416\n",
      "Number of processed documents: 4417\n",
      "Number of processed documents: 4418\n",
      "Number of processed documents: 4419\n",
      "Number of processed documents: 4420\n",
      "Number of processed documents: 4421\n",
      "Number of processed documents: 4422\n",
      "Number of processed documents: 4423\n",
      "Number of processed documents: 4424\n",
      "Number of processed documents: 4425\n",
      "Number of processed documents: 4426\n",
      "Number of processed documents: 4427\n",
      "Number of processed documents: 4428\n",
      "Number of processed documents: 4429\n",
      "Number of processed documents: 4430\n",
      "Number of processed documents: 4431\n",
      "Number of processed documents: 4432\n",
      "Number of processed documents: 4433\n",
      "Number of processed documents: 4434\n",
      "Number of processed documents: 4435\n",
      "Number of processed documents: 4436\n",
      "Number of processed documents: 4437\n",
      "Number of processed documents: 4438\n",
      "Number of processed documents: 4439\n",
      "Number of processed documents: 4440\n",
      "Number of processed documents: 4441\n",
      "Number of processed documents: 4442\n",
      "Number of processed documents: 4443\n",
      "Number of processed documents: 4444\n",
      "Number of processed documents: 4445\n",
      "Number of processed documents: 4446\n",
      "Number of processed documents: 4447\n",
      "Number of processed documents: 4448\n",
      "Number of processed documents: 4449\n",
      "Number of processed documents: 4450\n",
      "Number of processed documents: 4451\n",
      "Number of processed documents: 4452\n",
      "Number of processed documents: 4453\n",
      "Number of processed documents: 4454\n",
      "Number of processed documents: 4455\n",
      "Number of processed documents: 4456\n",
      "Number of processed documents: 4457\n",
      "Number of processed documents: 4458\n",
      "Number of processed documents: 4459\n",
      "Number of processed documents: 4460\n",
      "Number of processed documents: 4461\n",
      "Number of processed documents: 4462\n",
      "Number of processed documents: 4463\n",
      "Number of processed documents: 4464\n",
      "Number of processed documents: 4465\n",
      "Number of processed documents: 4466\n",
      "Number of processed documents: 4467\n",
      "Number of processed documents: 4468\n",
      "Number of processed documents: 4469\n",
      "Number of processed documents: 4470\n",
      "Number of processed documents: 4471\n",
      "Number of processed documents: 4472\n",
      "Number of processed documents: 4473\n",
      "Number of processed documents: 4474\n",
      "Number of processed documents: 4475\n",
      "Number of processed documents: 4476\n",
      "Number of processed documents: 4477\n",
      "Number of processed documents: 4478\n",
      "Number of processed documents: 4479\n",
      "Number of processed documents: 4480\n",
      "Number of processed documents: 4481\n",
      "Number of processed documents: 4482\n",
      "Number of processed documents: 4483\n",
      "Number of processed documents: 4484\n",
      "Number of processed documents: 4485\n",
      "Number of processed documents: 4486\n",
      "Number of processed documents: 4487\n",
      "Number of processed documents: 4488\n",
      "Number of processed documents: 4489\n",
      "Number of processed documents: 4490\n",
      "Number of processed documents: 4491\n",
      "Number of processed documents: 4492\n",
      "Number of processed documents: 4493\n",
      "Number of processed documents: 4494\n",
      "Number of processed documents: 4495\n",
      "Number of processed documents: 4496\n",
      "Number of processed documents: 4497\n",
      "Number of processed documents: 4498\n",
      "Number of processed documents: 4499\n",
      "Number of processed documents: 4500\n",
      "Number of processed documents: 4501\n",
      "Number of processed documents: 4502\n",
      "Number of processed documents: 4503\n",
      "Number of processed documents: 4504\n",
      "Number of processed documents: 4505\n",
      "Number of processed documents: 4506\n",
      "Number of processed documents: 4507\n",
      "Number of processed documents: 4508\n",
      "Number of processed documents: 4509\n",
      "Number of processed documents: 4510\n",
      "Number of processed documents: 4511\n",
      "Number of processed documents: 4512\n",
      "Number of processed documents: 4513\n",
      "Number of processed documents: 4514\n",
      "Number of processed documents: 4515\n",
      "Number of processed documents: 4516\n",
      "Number of processed documents: 4517\n",
      "Number of processed documents: 4518\n",
      "Number of processed documents: 4519\n",
      "Number of processed documents: 4520\n",
      "Number of processed documents: 4521\n",
      "Number of processed documents: 4522\n",
      "Number of processed documents: 4523\n",
      "Number of processed documents: 4524\n",
      "Number of processed documents: 4525\n",
      "Number of processed documents: 4526\n",
      "Number of processed documents: 4527\n",
      "Number of processed documents: 4528\n",
      "Number of processed documents: 4529\n",
      "Number of processed documents: 4530\n",
      "Number of processed documents: 4531\n",
      "Number of processed documents: 4532\n",
      "Number of processed documents: 4533\n",
      "Number of processed documents: 4534\n",
      "Number of processed documents: 4535\n",
      "Number of processed documents: 4536\n",
      "Number of processed documents: 4537\n",
      "Number of processed documents: 4538\n",
      "Number of processed documents: 4539\n",
      "Number of processed documents: 4540\n",
      "Number of processed documents: 4541\n",
      "Number of processed documents: 4542\n",
      "Number of processed documents: 4543\n",
      "Number of processed documents: 4544\n",
      "Number of processed documents: 4545\n",
      "Number of processed documents: 4546\n",
      "Number of processed documents: 4547\n",
      "Number of processed documents: 4548\n",
      "Number of processed documents: 4549\n",
      "Number of processed documents: 4550\n",
      "Number of processed documents: 4551\n",
      "Number of processed documents: 4552\n",
      "Number of processed documents: 4553\n",
      "Number of processed documents: 4554\n",
      "Number of processed documents: 4555\n",
      "Number of processed documents: 4556\n",
      "Number of processed documents: 4557\n",
      "Number of processed documents: 4558\n",
      "Number of processed documents: 4559\n",
      "Number of processed documents: 4560\n",
      "Number of processed documents: 4561\n",
      "Number of processed documents: 4562\n",
      "Number of processed documents: 4563\n",
      "Number of processed documents: 4564\n",
      "Number of processed documents: 4565\n",
      "Number of processed documents: 4566\n",
      "Number of processed documents: 4567\n",
      "Number of processed documents: 4568\n",
      "Number of processed documents: 4569\n",
      "Number of processed documents: 4570\n",
      "Number of processed documents: 4571\n",
      "Number of processed documents: 4572\n",
      "Number of processed documents: 4573\n",
      "Number of processed documents: 4574\n",
      "Number of processed documents: 4575\n",
      "Number of processed documents: 4576\n",
      "Number of processed documents: 4577\n",
      "Number of processed documents: 4578\n",
      "Number of processed documents: 4579\n",
      "Number of processed documents: 4580\n",
      "Number of processed documents: 4581\n",
      "Number of processed documents: 4582\n",
      "Number of processed documents: 4583\n",
      "Number of processed documents: 4584\n",
      "Number of processed documents: 4585\n",
      "Number of processed documents: 4586\n",
      "Number of processed documents: 4587\n",
      "Number of processed documents: 4588\n",
      "Number of processed documents: 4589\n",
      "Number of processed documents: 4590\n",
      "Number of processed documents: 4591\n",
      "Number of processed documents: 4592\n",
      "Number of processed documents: 4593\n",
      "Number of processed documents: 4594\n",
      "Number of processed documents: 4595\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 4596\n",
      "Number of processed documents: 4597\n",
      "Number of processed documents: 4598\n",
      "Number of processed documents: 4599\n",
      "Number of processed documents: 4600\n",
      "Number of processed documents: 4601\n",
      "Number of processed documents: 4602\n",
      "Number of processed documents: 4603\n",
      "Number of processed documents: 4604\n",
      "Number of processed documents: 4605\n",
      "Number of processed documents: 4606\n",
      "Number of processed documents: 4607\n",
      "Number of processed documents: 4608\n",
      "Number of processed documents: 4609\n",
      "Number of processed documents: 4610\n",
      "Number of processed documents: 4611\n",
      "Number of processed documents: 4612\n",
      "Number of processed documents: 4613\n",
      "Number of processed documents: 4614\n",
      "Number of processed documents: 4615\n",
      "Number of processed documents: 4616\n",
      "Number of processed documents: 4617\n",
      "Number of processed documents: 4618\n",
      "Number of processed documents: 4619\n",
      "Number of processed documents: 4620\n",
      "Number of processed documents: 4621\n",
      "Number of processed documents: 4622\n",
      "Number of processed documents: 4623\n",
      "Number of processed documents: 4624\n",
      "Number of processed documents: 4625\n",
      "Number of processed documents: 4626\n",
      "Number of processed documents: 4627\n",
      "Number of processed documents: 4628\n",
      "Number of processed documents: 4629\n",
      "Number of processed documents: 4630\n",
      "Number of processed documents: 4631\n",
      "Number of processed documents: 4632\n",
      "Number of processed documents: 4633\n",
      "Number of processed documents: 4634\n",
      "Number of processed documents: 4635\n",
      "Number of processed documents: 4636\n",
      "Number of processed documents: 4637\n",
      "Number of processed documents: 4638\n",
      "Number of processed documents: 4639\n",
      "Number of processed documents: 4640\n",
      "Number of processed documents: 4641\n",
      "Number of processed documents: 4642\n",
      "Number of processed documents: 4643\n",
      "Number of processed documents: 4644\n",
      "Number of processed documents: 4645\n",
      "Number of processed documents: 4646\n",
      "Number of processed documents: 4647\n",
      "Number of processed documents: 4648\n",
      "Number of processed documents: 4649\n",
      "Number of processed documents: 4650\n",
      "Number of processed documents: 4651\n",
      "Number of processed documents: 4652\n",
      "Number of processed documents: 4653\n",
      "Number of processed documents: 4654\n",
      "Number of processed documents: 4655\n",
      "Number of processed documents: 4656\n",
      "Number of processed documents: 4657\n",
      "Number of processed documents: 4658\n",
      "Number of processed documents: 4659\n",
      "Number of processed documents: 4660\n",
      "Number of processed documents: 4661\n",
      "Number of processed documents: 4662\n",
      "Number of processed documents: 4663\n",
      "Number of processed documents: 4664\n",
      "Number of processed documents: 4665\n",
      "Number of processed documents: 4666\n",
      "Number of processed documents: 4667\n",
      "Number of processed documents: 4668\n",
      "Number of processed documents: 4669\n",
      "Number of processed documents: 4670\n",
      "Number of processed documents: 4671\n",
      "Number of processed documents: 4672\n",
      "Number of processed documents: 4673\n",
      "Number of processed documents: 4674\n",
      "Number of processed documents: 4675\n",
      "Number of processed documents: 4676\n",
      "Number of processed documents: 4677\n",
      "Number of processed documents: 4678\n",
      "Number of processed documents: 4679\n",
      "Number of processed documents: 4680\n",
      "Number of processed documents: 4681\n",
      "Number of processed documents: 4682\n",
      "Number of processed documents: 4683\n",
      "Number of processed documents: 4684\n",
      "Number of processed documents: 4685\n",
      "Number of processed documents: 4686\n",
      "Number of processed documents: 4687\n",
      "Number of processed documents: 4688\n",
      "Number of processed documents: 4689\n",
      "Number of processed documents: 4690\n",
      "Number of processed documents: 4691\n",
      "Number of processed documents: 4692\n",
      "Number of processed documents: 4693\n",
      "Number of processed documents: 4694\n",
      "Number of processed documents: 4695\n",
      "Number of processed documents: 4696\n",
      "Number of processed documents: 4697\n",
      "Number of processed documents: 4698\n",
      "Number of processed documents: 4699\n",
      "Number of processed documents: 4700\n",
      "Number of processed documents: 4701\n",
      "Number of processed documents: 4702\n",
      "Number of processed documents: 4703\n",
      "Number of processed documents: 4704\n",
      "Number of processed documents: 4705\n",
      "Number of processed documents: 4706\n",
      "Number of processed documents: 4707\n",
      "Number of processed documents: 4708\n",
      "Number of processed documents: 4709\n",
      "Number of processed documents: 4710\n",
      "Number of processed documents: 4711\n",
      "Number of processed documents: 4712\n",
      "Number of processed documents: 4713\n",
      "Number of processed documents: 4714\n",
      "Number of processed documents: 4715\n",
      "Number of processed documents: 4716\n",
      "Number of processed documents: 4717\n",
      "Number of processed documents: 4718\n",
      "Number of processed documents: 4719\n",
      "Number of processed documents: 4720\n",
      "Number of processed documents: 4721\n",
      "Number of processed documents: 4722\n",
      "Number of processed documents: 4723\n",
      "Number of processed documents: 4724\n",
      "Number of processed documents: 4725\n",
      "Number of processed documents: 4726\n",
      "Number of processed documents: 4727\n",
      "Number of processed documents: 4728\n",
      "Number of processed documents: 4729\n",
      "Number of processed documents: 4730\n",
      "Number of processed documents: 4731\n",
      "Number of processed documents: 4732\n",
      "Number of processed documents: 4733\n",
      "Number of processed documents: 4734\n",
      "Number of processed documents: 4735\n",
      "Number of processed documents: 4736\n",
      "Number of processed documents: 4737\n",
      "Number of processed documents: 4738\n",
      "Number of processed documents: 4739\n",
      "Number of processed documents: 4740\n",
      "Number of processed documents: 4741\n",
      "Number of processed documents: 4742\n",
      "Number of processed documents: 4743\n",
      "Number of processed documents: 4744\n",
      "Number of processed documents: 4745\n",
      "Number of processed documents: 4746\n",
      "Number of processed documents: 4747\n",
      "Number of processed documents: 4748\n",
      "Number of processed documents: 4749\n",
      "Number of processed documents: 4750\n",
      "Number of processed documents: 4751\n",
      "Number of processed documents: 4752\n",
      "Number of processed documents: 4753\n",
      "Number of processed documents: 4754\n",
      "Number of processed documents: 4755\n",
      "Number of processed documents: 4756\n",
      "Number of processed documents: 4757\n",
      "Number of processed documents: 4758\n",
      "Number of processed documents: 4759\n",
      "Number of processed documents: 4760\n",
      "Number of processed documents: 4761\n",
      "Number of processed documents: 4762\n",
      "Number of processed documents: 4763\n",
      "Number of processed documents: 4764\n",
      "Number of processed documents: 4765\n",
      "Number of processed documents: 4766\n",
      "Number of processed documents: 4767\n",
      "Number of processed documents: 4768\n",
      "Number of processed documents: 4769\n",
      "Number of processed documents: 4770\n",
      "Number of processed documents: 4771\n",
      "Number of processed documents: 4772\n",
      "Number of processed documents: 4773\n",
      "Number of processed documents: 4774\n",
      "Number of processed documents: 4775\n",
      "Number of processed documents: 4776\n",
      "Number of processed documents: 4777\n",
      "Number of processed documents: 4778\n",
      "Number of processed documents: 4779\n",
      "Number of processed documents: 4780\n",
      "Number of processed documents: 4781\n",
      "Number of processed documents: 4782\n",
      "Number of processed documents: 4783\n",
      "Number of processed documents: 4784\n",
      "Number of processed documents: 4785\n",
      "Number of processed documents: 4786\n",
      "Number of processed documents: 4787\n",
      "Number of processed documents: 4788\n",
      "Number of processed documents: 4789\n",
      "Number of processed documents: 4790\n",
      "Number of processed documents: 4791\n",
      "Number of processed documents: 4792\n",
      "Number of processed documents: 4793\n",
      "Number of processed documents: 4794\n",
      "Number of processed documents: 4795\n",
      "Number of processed documents: 4796\n",
      "Number of processed documents: 4797\n",
      "Number of processed documents: 4798\n",
      "Number of processed documents: 4799\n",
      "Number of processed documents: 4800\n",
      "Number of processed documents: 4801\n",
      "Number of processed documents: 4802\n",
      "Number of processed documents: 4803\n",
      "Number of processed documents: 4804\n",
      "Number of processed documents: 4805\n",
      "Number of processed documents: 4806\n",
      "Number of processed documents: 4807\n",
      "Number of processed documents: 4808\n",
      "Number of processed documents: 4809\n",
      "Number of processed documents: 4810\n",
      "Number of processed documents: 4811\n",
      "Number of processed documents: 4812\n",
      "Number of processed documents: 4813\n",
      "Number of processed documents: 4814\n",
      "Number of processed documents: 4815\n",
      "Number of processed documents: 4816\n",
      "Number of processed documents: 4817\n",
      "Number of processed documents: 4818\n",
      "Number of processed documents: 4819\n",
      "Number of processed documents: 4820\n",
      "Number of processed documents: 4821\n",
      "Number of processed documents: 4822\n",
      "Number of processed documents: 4823\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 4824\n",
      "Number of processed documents: 4825\n",
      "Number of processed documents: 4826\n",
      "Number of processed documents: 4827\n",
      "Number of processed documents: 4828\n",
      "Number of processed documents: 4829\n",
      "Number of processed documents: 4830\n",
      "Number of processed documents: 4831\n",
      "Number of processed documents: 4832\n",
      "Number of processed documents: 4833\n",
      "Number of processed documents: 4834\n",
      "Number of processed documents: 4835\n",
      "Number of processed documents: 4836\n",
      "Number of processed documents: 4837\n",
      "Number of processed documents: 4838\n",
      "Number of processed documents: 4839\n",
      "Number of processed documents: 4840\n",
      "Number of processed documents: 4841\n",
      "Number of processed documents: 4842\n",
      "Number of processed documents: 4843\n",
      "Number of processed documents: 4844\n",
      "Number of processed documents: 4845\n",
      "Number of processed documents: 4846\n",
      "Number of processed documents: 4847\n",
      "Number of processed documents: 4848\n",
      "Number of processed documents: 4849\n",
      "Number of processed documents: 4850\n",
      "Number of processed documents: 4851\n",
      "Number of processed documents: 4852\n",
      "Number of processed documents: 4853\n",
      "Number of processed documents: 4854\n",
      "Number of processed documents: 4855\n",
      "Number of processed documents: 4856\n",
      "Number of processed documents: 4857\n",
      "Number of processed documents: 4858\n",
      "Number of processed documents: 4859\n",
      "Number of processed documents: 4860\n",
      "Number of processed documents: 4861\n",
      "Number of processed documents: 4862\n",
      "Number of processed documents: 4863\n",
      "Number of processed documents: 4864\n",
      "Number of processed documents: 4865\n",
      "Number of processed documents: 4866\n",
      "Number of processed documents: 4867\n",
      "Number of processed documents: 4868\n",
      "Number of processed documents: 4869\n",
      "Number of processed documents: 4870\n",
      "Number of processed documents: 4871\n",
      "Number of processed documents: 4872\n",
      "Number of processed documents: 4873\n",
      "Number of processed documents: 4874\n",
      "Number of processed documents: 4875\n",
      "Number of processed documents: 4876\n",
      "Number of processed documents: 4877\n",
      "Number of processed documents: 4878\n",
      "Number of processed documents: 4879\n",
      "Number of processed documents: 4880\n",
      "Number of processed documents: 4881\n",
      "Number of processed documents: 4882\n",
      "Number of processed documents: 4883\n",
      "Number of processed documents: 4884\n",
      "Number of processed documents: 4885\n",
      "Number of processed documents: 4886\n",
      "Number of processed documents: 4887\n",
      "Number of processed documents: 4888\n",
      "Number of processed documents: 4889\n",
      "Number of processed documents: 4890\n",
      "Number of processed documents: 4891\n",
      "Number of processed documents: 4892\n",
      "Number of processed documents: 4893\n",
      "Number of processed documents: 4894\n",
      "Number of processed documents: 4895\n",
      "Number of processed documents: 4896\n",
      "Number of processed documents: 4897\n",
      "Number of processed documents: 4898\n",
      "Number of processed documents: 4899\n",
      "Number of processed documents: 4900\n",
      "Number of processed documents: 4901\n",
      "Number of processed documents: 4902\n",
      "Number of processed documents: 4903\n",
      "Number of processed documents: 4904\n",
      "Number of processed documents: 4905\n",
      "Number of processed documents: 4906\n",
      "Number of processed documents: 4907\n",
      "Number of processed documents: 4908\n",
      "Number of processed documents: 4909\n",
      "Number of processed documents: 4910\n",
      "Number of processed documents: 4911\n",
      "Number of processed documents: 4912\n",
      "Number of processed documents: 4913\n",
      "Number of processed documents: 4914\n",
      "Number of processed documents: 4915\n",
      "Number of processed documents: 4916\n",
      "Number of processed documents: 4917\n",
      "Number of processed documents: 4918\n",
      "Number of processed documents: 4919\n",
      "Number of processed documents: 4920\n",
      "Number of processed documents: 4921\n",
      "Number of processed documents: 4922\n",
      "Number of processed documents: 4923\n",
      "Number of processed documents: 4924\n",
      "Number of processed documents: 4925\n",
      "Number of processed documents: 4926\n",
      "Number of processed documents: 4927\n",
      "Number of processed documents: 4928\n",
      "Number of processed documents: 4929\n",
      "Number of processed documents: 4930\n",
      "Number of processed documents: 4931\n",
      "Number of processed documents: 4932\n",
      "Number of processed documents: 4933\n",
      "Number of processed documents: 4934\n",
      "Number of processed documents: 4935\n",
      "Number of processed documents: 4936\n",
      "Number of processed documents: 4937\n",
      "Number of processed documents: 4938\n",
      "Number of processed documents: 4939\n",
      "Number of processed documents: 4940\n",
      "Number of processed documents: 4941\n",
      "Number of processed documents: 4942\n",
      "Number of processed documents: 4943\n",
      "Number of processed documents: 4944\n",
      "Number of processed documents: 4945\n",
      "Number of processed documents: 4946\n",
      "Number of processed documents: 4947\n",
      "Number of processed documents: 4948\n",
      "Number of processed documents: 4949\n",
      "Number of processed documents: 4950\n",
      "Number of processed documents: 4951\n",
      "Number of processed documents: 4952\n",
      "Number of processed documents: 4953\n",
      "Number of processed documents: 4954\n",
      "Number of processed documents: 4955\n",
      "Number of processed documents: 4956\n",
      "Number of processed documents: 4957\n",
      "Number of processed documents: 4958\n",
      "Number of processed documents: 4959\n",
      "Number of processed documents: 4960\n",
      "Number of processed documents: 4961\n",
      "Number of processed documents: 4962\n",
      "Number of processed documents: 4963\n",
      "Number of processed documents: 4964\n",
      "Number of processed documents: 4965\n",
      "Number of processed documents: 4966\n",
      "Number of processed documents: 4967\n",
      "Number of processed documents: 4968\n",
      "Number of processed documents: 4969\n",
      "Number of processed documents: 4970\n",
      "Number of processed documents: 4971\n",
      "Number of processed documents: 4972\n",
      "Number of processed documents: 4973\n",
      "Number of processed documents: 4974\n",
      "Number of processed documents: 4975\n",
      "Number of processed documents: 4976\n",
      "Number of processed documents: 4977\n",
      "Number of processed documents: 4978\n",
      "Number of processed documents: 4979\n",
      "Number of processed documents: 4980\n",
      "Number of processed documents: 4981\n",
      "Number of processed documents: 4982\n",
      "Number of processed documents: 4983\n",
      "Number of processed documents: 4984\n",
      "Number of processed documents: 4985\n",
      "Number of processed documents: 4986\n",
      "Number of processed documents: 4987\n",
      "Number of processed documents: 4988\n",
      "Number of processed documents: 4989\n",
      "Number of processed documents: 4990\n",
      "Number of processed documents: 4991\n",
      "Number of processed documents: 4992\n",
      "Number of processed documents: 4993\n",
      "Number of processed documents: 4994\n",
      "Number of processed documents: 4995\n",
      "Number of processed documents: 4996\n",
      "Number of processed documents: 4997\n",
      "Number of processed documents: 4998\n",
      "Number of processed documents: 4999\n",
      "Number of processed documents: 5000\n",
      "Number of processed documents: 5001\n",
      "Number of processed documents: 5002\n",
      "Number of processed documents: 5003\n",
      "Number of processed documents: 5004\n",
      "Number of processed documents: 5005\n",
      "Number of processed documents: 5006\n",
      "Number of processed documents: 5007\n",
      "Number of processed documents: 5008\n",
      "Number of processed documents: 5009\n",
      "Number of processed documents: 5010\n",
      "Number of processed documents: 5011\n",
      "Number of processed documents: 5012\n",
      "Number of processed documents: 5013\n",
      "Number of processed documents: 5014\n",
      "Number of processed documents: 5015\n",
      "Number of processed documents: 5016\n",
      "Number of processed documents: 5017\n",
      "Number of processed documents: 5018\n",
      "Number of processed documents: 5019\n",
      "Number of processed documents: 5020\n",
      "Number of processed documents: 5021\n",
      "Number of processed documents: 5022\n",
      "Number of processed documents: 5023\n",
      "Number of processed documents: 5024\n",
      "Number of processed documents: 5025\n",
      "Number of processed documents: 5026\n",
      "Number of processed documents: 5027\n",
      "Number of processed documents: 5028\n",
      "Number of processed documents: 5029\n",
      "Number of processed documents: 5030\n",
      "Number of processed documents: 5031\n",
      "Number of processed documents: 5032\n",
      "Number of processed documents: 5033\n",
      "Number of processed documents: 5034\n",
      "Number of processed documents: 5035\n",
      "Number of processed documents: 5036\n",
      "Number of processed documents: 5037\n",
      "Number of processed documents: 5038\n",
      "Number of processed documents: 5039\n",
      "Number of processed documents: 5040\n",
      "Number of processed documents: 5041\n",
      "Number of processed documents: 5042\n",
      "Number of processed documents: 5043\n",
      "Number of processed documents: 5044\n",
      "Number of processed documents: 5045\n",
      "Number of processed documents: 5046\n",
      "Number of processed documents: 5047\n",
      "Number of processed documents: 5048\n",
      "Number of processed documents: 5049\n",
      "Number of processed documents: 5050\n",
      "Number of processed documents: 5051\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 5052\n",
      "Number of processed documents: 5053\n",
      "Number of processed documents: 5054\n",
      "Number of processed documents: 5055\n",
      "Number of processed documents: 5056\n",
      "Number of processed documents: 5057\n",
      "Number of processed documents: 5058\n",
      "Number of processed documents: 5059\n",
      "Number of processed documents: 5060\n",
      "Number of processed documents: 5061\n",
      "Number of processed documents: 5062\n",
      "Number of processed documents: 5063\n",
      "Number of processed documents: 5064\n",
      "Number of processed documents: 5065\n",
      "Number of processed documents: 5066\n",
      "Number of processed documents: 5067\n",
      "Number of processed documents: 5068\n",
      "Number of processed documents: 5069\n",
      "Number of processed documents: 5070\n",
      "Number of processed documents: 5071\n",
      "Number of processed documents: 5072\n",
      "Number of processed documents: 5073\n",
      "Number of processed documents: 5074\n",
      "Number of processed documents: 5075\n",
      "Number of processed documents: 5076\n",
      "Number of processed documents: 5077\n",
      "Number of processed documents: 5078\n",
      "Number of processed documents: 5079\n",
      "Number of processed documents: 5080\n",
      "Number of processed documents: 5081\n",
      "Number of processed documents: 5082\n",
      "Number of processed documents: 5083\n",
      "Number of processed documents: 5084\n",
      "Number of processed documents: 5085\n",
      "Number of processed documents: 5086\n",
      "Number of processed documents: 5087\n",
      "Number of processed documents: 5088\n",
      "Number of processed documents: 5089\n",
      "Number of processed documents: 5090\n",
      "Number of processed documents: 5091\n",
      "Number of processed documents: 5092\n",
      "Number of processed documents: 5093\n",
      "Number of processed documents: 5094\n",
      "Number of processed documents: 5095\n",
      "Number of processed documents: 5096\n",
      "Number of processed documents: 5097\n",
      "Number of processed documents: 5098\n",
      "Number of processed documents: 5099\n",
      "Number of processed documents: 5100\n",
      "Number of processed documents: 5101\n",
      "Number of processed documents: 5102\n",
      "Number of processed documents: 5103\n",
      "Number of processed documents: 5104\n",
      "Number of processed documents: 5105\n",
      "Number of processed documents: 5106\n",
      "Number of processed documents: 5107\n",
      "Number of processed documents: 5108\n",
      "Number of processed documents: 5109\n",
      "Number of processed documents: 5110\n",
      "Number of processed documents: 5111\n",
      "Number of processed documents: 5112\n",
      "Number of processed documents: 5113\n",
      "Number of processed documents: 5114\n",
      "Number of processed documents: 5115\n",
      "Number of processed documents: 5116\n",
      "Number of processed documents: 5117\n",
      "Number of processed documents: 5118\n",
      "Number of processed documents: 5119\n",
      "Number of processed documents: 5120\n",
      "Number of processed documents: 5121\n",
      "Number of processed documents: 5122\n",
      "Number of processed documents: 5123\n",
      "Number of processed documents: 5124\n",
      "Number of processed documents: 5125\n",
      "Number of processed documents: 5126\n",
      "Number of processed documents: 5127\n",
      "Number of processed documents: 5128\n",
      "Number of processed documents: 5129\n",
      "Number of processed documents: 5130\n",
      "Number of processed documents: 5131\n",
      "Number of processed documents: 5132\n",
      "Number of processed documents: 5133\n",
      "Number of processed documents: 5134\n",
      "Number of processed documents: 5135\n",
      "Number of processed documents: 5136\n",
      "Number of processed documents: 5137\n",
      "Number of processed documents: 5138\n",
      "Number of processed documents: 5139\n",
      "Number of processed documents: 5140\n",
      "Number of processed documents: 5141\n",
      "Number of processed documents: 5142\n",
      "Number of processed documents: 5143\n",
      "Number of processed documents: 5144\n",
      "Number of processed documents: 5145\n",
      "Number of processed documents: 5146\n",
      "Number of processed documents: 5147\n",
      "Number of processed documents: 5148\n",
      "Number of processed documents: 5149\n",
      "Number of processed documents: 5150\n",
      "Number of processed documents: 5151\n",
      "Number of processed documents: 5152\n",
      "Number of processed documents: 5153\n",
      "Number of processed documents: 5154\n",
      "Number of processed documents: 5155\n",
      "Number of processed documents: 5156\n",
      "Number of processed documents: 5157\n",
      "Number of processed documents: 5158\n",
      "Number of processed documents: 5159\n",
      "Number of processed documents: 5160\n",
      "Number of processed documents: 5161\n",
      "Number of processed documents: 5162\n",
      "Number of processed documents: 5163\n",
      "Number of processed documents: 5164\n",
      "Number of processed documents: 5165\n",
      "Number of processed documents: 5166\n",
      "Number of processed documents: 5167\n",
      "Number of processed documents: 5168\n",
      "Number of processed documents: 5169\n",
      "Number of processed documents: 5170\n",
      "Number of processed documents: 5171\n",
      "Number of processed documents: 5172\n",
      "Number of processed documents: 5173\n",
      "Number of processed documents: 5174\n",
      "Number of processed documents: 5175\n",
      "Number of processed documents: 5176\n",
      "Number of processed documents: 5177\n",
      "Number of processed documents: 5178\n",
      "Number of processed documents: 5179\n",
      "Number of processed documents: 5180\n",
      "Number of processed documents: 5181\n",
      "Number of processed documents: 5182\n",
      "Number of processed documents: 5183\n",
      "Number of processed documents: 5184\n",
      "Number of processed documents: 5185\n",
      "Number of processed documents: 5186\n",
      "Number of processed documents: 5187\n",
      "Number of processed documents: 5188\n",
      "Number of processed documents: 5189\n",
      "Number of processed documents: 5190\n",
      "Number of processed documents: 5191\n",
      "Number of processed documents: 5192\n",
      "Number of processed documents: 5193\n",
      "Number of processed documents: 5194\n",
      "Number of processed documents: 5195\n",
      "Number of processed documents: 5196\n",
      "Number of processed documents: 5197\n",
      "Number of processed documents: 5198\n",
      "Number of processed documents: 5199\n",
      "Number of processed documents: 5200\n",
      "Number of processed documents: 5201\n",
      "Number of processed documents: 5202\n",
      "Number of processed documents: 5203\n",
      "Number of processed documents: 5204\n",
      "Number of processed documents: 5205\n",
      "Number of processed documents: 5206\n",
      "Number of processed documents: 5207\n",
      "Number of processed documents: 5208\n",
      "Number of processed documents: 5209\n",
      "Number of processed documents: 5210\n",
      "Number of processed documents: 5211\n",
      "Number of processed documents: 5212\n",
      "Number of processed documents: 5213\n",
      "Number of processed documents: 5214\n",
      "Number of processed documents: 5215\n",
      "Number of processed documents: 5216\n",
      "Number of processed documents: 5217\n",
      "Number of processed documents: 5218\n",
      "Number of processed documents: 5219\n",
      "Number of processed documents: 5220\n",
      "Number of processed documents: 5221\n",
      "Number of processed documents: 5222\n",
      "Number of processed documents: 5223\n",
      "Number of processed documents: 5224\n",
      "Number of processed documents: 5225\n",
      "Number of processed documents: 5226\n",
      "Number of processed documents: 5227\n",
      "Number of processed documents: 5228\n",
      "Number of processed documents: 5229\n",
      "Number of processed documents: 5230\n",
      "Number of processed documents: 5231\n",
      "Number of processed documents: 5232\n",
      "Number of processed documents: 5233\n",
      "Number of processed documents: 5234\n",
      "Number of processed documents: 5235\n",
      "Number of processed documents: 5236\n",
      "Number of processed documents: 5237\n",
      "Number of processed documents: 5238\n",
      "Number of processed documents: 5239\n",
      "Number of processed documents: 5240\n",
      "Number of processed documents: 5241\n",
      "Number of processed documents: 5242\n",
      "Number of processed documents: 5243\n",
      "Number of processed documents: 5244\n",
      "Number of processed documents: 5245\n",
      "Number of processed documents: 5246\n",
      "Number of processed documents: 5247\n",
      "Number of processed documents: 5248\n",
      "Number of processed documents: 5249\n",
      "Number of processed documents: 5250\n",
      "Number of processed documents: 5251\n",
      "Number of processed documents: 5252\n",
      "Number of processed documents: 5253\n",
      "Number of processed documents: 5254\n",
      "Number of processed documents: 5255\n",
      "Number of processed documents: 5256\n",
      "Number of processed documents: 5257\n",
      "Number of processed documents: 5258\n",
      "Number of processed documents: 5259\n",
      "Number of processed documents: 5260\n",
      "Number of processed documents: 5261\n",
      "Number of processed documents: 5262\n",
      "Number of processed documents: 5263\n",
      "Number of processed documents: 5264\n",
      "Number of processed documents: 5265\n",
      "Number of processed documents: 5266\n",
      "Number of processed documents: 5267\n",
      "Number of processed documents: 5268\n",
      "Number of processed documents: 5269\n",
      "Number of processed documents: 5270\n",
      "Number of processed documents: 5271\n",
      "Number of processed documents: 5272\n",
      "Number of processed documents: 5273\n",
      "Number of processed documents: 5274\n",
      "Number of processed documents: 5275\n",
      "Number of processed documents: 5276\n",
      "Number of processed documents: 5277\n",
      "Number of processed documents: 5278\n",
      "Number of processed documents: 5279\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 5280\n",
      "Number of processed documents: 5281\n",
      "Number of processed documents: 5282\n",
      "Number of processed documents: 5283\n",
      "Number of processed documents: 5284\n",
      "Number of processed documents: 5285\n",
      "Number of processed documents: 5286\n",
      "Number of processed documents: 5287\n",
      "Number of processed documents: 5288\n",
      "Number of processed documents: 5289\n",
      "Number of processed documents: 5290\n",
      "Number of processed documents: 5291\n",
      "Number of processed documents: 5292\n",
      "Number of processed documents: 5293\n",
      "Number of processed documents: 5294\n",
      "Number of processed documents: 5295\n",
      "Number of processed documents: 5296\n",
      "Number of processed documents: 5297\n",
      "Number of processed documents: 5298\n",
      "Number of processed documents: 5299\n",
      "Number of processed documents: 5300\n",
      "Number of processed documents: 5301\n",
      "Number of processed documents: 5302\n",
      "Number of processed documents: 5303\n",
      "Number of processed documents: 5304\n",
      "Number of processed documents: 5305\n",
      "Number of processed documents: 5306\n",
      "Number of processed documents: 5307\n",
      "Number of processed documents: 5308\n",
      "Number of processed documents: 5309\n",
      "Number of processed documents: 5310\n",
      "Number of processed documents: 5311\n",
      "Number of processed documents: 5312\n",
      "Number of processed documents: 5313\n",
      "Number of processed documents: 5314\n",
      "Number of processed documents: 5315\n",
      "Number of processed documents: 5316\n",
      "Number of processed documents: 5317\n",
      "Number of processed documents: 5318\n",
      "Number of processed documents: 5319\n",
      "Number of processed documents: 5320\n",
      "Number of processed documents: 5321\n",
      "Number of processed documents: 5322\n",
      "Number of processed documents: 5323\n",
      "Number of processed documents: 5324\n",
      "Number of processed documents: 5325\n",
      "Number of processed documents: 5326\n",
      "Number of processed documents: 5327\n",
      "Number of processed documents: 5328\n",
      "Number of processed documents: 5329\n",
      "Number of processed documents: 5330\n",
      "Number of processed documents: 5331\n",
      "Number of processed documents: 5332\n",
      "Number of processed documents: 5333\n",
      "Number of processed documents: 5334\n",
      "Number of processed documents: 5335\n",
      "Number of processed documents: 5336\n",
      "Number of processed documents: 5337\n",
      "Number of processed documents: 5338\n",
      "Number of processed documents: 5339\n",
      "Number of processed documents: 5340\n",
      "Number of processed documents: 5341\n",
      "Number of processed documents: 5342\n",
      "Number of processed documents: 5343\n",
      "Number of processed documents: 5344\n",
      "Number of processed documents: 5345\n",
      "Number of processed documents: 5346\n",
      "Number of processed documents: 5347\n",
      "Number of processed documents: 5348\n",
      "Number of processed documents: 5349\n",
      "Number of processed documents: 5350\n",
      "Number of processed documents: 5351\n",
      "Number of processed documents: 5352\n",
      "Number of processed documents: 5353\n",
      "Number of processed documents: 5354\n",
      "Number of processed documents: 5355\n",
      "Number of processed documents: 5356\n",
      "Number of processed documents: 5357\n",
      "Number of processed documents: 5358\n",
      "Number of processed documents: 5359\n",
      "Number of processed documents: 5360\n",
      "Number of processed documents: 5361\n",
      "Number of processed documents: 5362\n",
      "Number of processed documents: 5363\n",
      "Number of processed documents: 5364\n",
      "Number of processed documents: 5365\n",
      "Number of processed documents: 5366\n",
      "Number of processed documents: 5367\n",
      "Number of processed documents: 5368\n",
      "Number of processed documents: 5369\n",
      "Number of processed documents: 5370\n",
      "Number of processed documents: 5371\n",
      "Number of processed documents: 5372\n",
      "Number of processed documents: 5373\n",
      "Number of processed documents: 5374\n",
      "Number of processed documents: 5375\n",
      "Number of processed documents: 5376\n",
      "Number of processed documents: 5377\n",
      "Number of processed documents: 5378\n",
      "Number of processed documents: 5379\n",
      "Number of processed documents: 5380\n",
      "Number of processed documents: 5381\n",
      "Number of processed documents: 5382\n",
      "Number of processed documents: 5383\n",
      "Number of processed documents: 5384\n",
      "Number of processed documents: 5385\n",
      "Number of processed documents: 5386\n",
      "Number of processed documents: 5387\n",
      "Number of processed documents: 5388\n",
      "Number of processed documents: 5389\n",
      "Number of processed documents: 5390\n",
      "Number of processed documents: 5391\n",
      "Number of processed documents: 5392\n",
      "Number of processed documents: 5393\n",
      "Number of processed documents: 5394\n",
      "Number of processed documents: 5395\n",
      "Number of processed documents: 5396\n",
      "Number of processed documents: 5397\n",
      "Number of processed documents: 5398\n",
      "Number of processed documents: 5399\n",
      "Number of processed documents: 5400\n",
      "Number of processed documents: 5401\n",
      "Number of processed documents: 5402\n",
      "Number of processed documents: 5403\n",
      "Number of processed documents: 5404\n",
      "Number of processed documents: 5405\n",
      "Number of processed documents: 5406\n",
      "Number of processed documents: 5407\n",
      "Number of processed documents: 5408\n",
      "Number of processed documents: 5409\n",
      "Number of processed documents: 5410\n",
      "Number of processed documents: 5411\n",
      "Number of processed documents: 5412\n",
      "Number of processed documents: 5413\n",
      "Number of processed documents: 5414\n",
      "Number of processed documents: 5415\n",
      "Number of processed documents: 5416\n",
      "Number of processed documents: 5417\n",
      "Number of processed documents: 5418\n",
      "Number of processed documents: 5419\n",
      "Number of processed documents: 5420\n",
      "Number of processed documents: 5421\n",
      "Number of processed documents: 5422\n",
      "Number of processed documents: 5423\n",
      "Number of processed documents: 5424\n",
      "Number of processed documents: 5425\n",
      "Number of processed documents: 5426\n",
      "Number of processed documents: 5427\n",
      "Number of processed documents: 5428\n",
      "Number of processed documents: 5429\n",
      "Number of processed documents: 5430\n",
      "Number of processed documents: 5431\n",
      "Number of processed documents: 5432\n",
      "Number of processed documents: 5433\n",
      "Number of processed documents: 5434\n",
      "Number of processed documents: 5435\n",
      "Number of processed documents: 5436\n",
      "Number of processed documents: 5437\n",
      "Number of processed documents: 5438\n",
      "Number of processed documents: 5439\n",
      "Number of processed documents: 5440\n",
      "Number of processed documents: 5441\n",
      "Number of processed documents: 5442\n",
      "Number of processed documents: 5443\n",
      "Number of processed documents: 5444\n",
      "Number of processed documents: 5445\n",
      "Number of processed documents: 5446\n",
      "Number of processed documents: 5447\n",
      "Number of processed documents: 5448\n",
      "Number of processed documents: 5449\n",
      "Number of processed documents: 5450\n",
      "Number of processed documents: 5451\n",
      "Number of processed documents: 5452\n",
      "Number of processed documents: 5453\n",
      "Number of processed documents: 5454\n",
      "Number of processed documents: 5455\n",
      "Number of processed documents: 5456\n",
      "Number of processed documents: 5457\n",
      "Number of processed documents: 5458\n",
      "Number of processed documents: 5459\n",
      "Number of processed documents: 5460\n",
      "Number of processed documents: 5461\n",
      "Number of processed documents: 5462\n",
      "Number of processed documents: 5463\n",
      "Number of processed documents: 5464\n",
      "Number of processed documents: 5465\n",
      "Number of processed documents: 5466\n",
      "Number of processed documents: 5467\n",
      "Number of processed documents: 5468\n",
      "Number of processed documents: 5469\n",
      "Number of processed documents: 5470\n",
      "Number of processed documents: 5471\n",
      "Number of processed documents: 5472\n",
      "Number of processed documents: 5473\n",
      "Number of processed documents: 5474\n",
      "Number of processed documents: 5475\n",
      "Number of processed documents: 5476\n",
      "Number of processed documents: 5477\n",
      "Number of processed documents: 5478\n",
      "Number of processed documents: 5479\n",
      "Number of processed documents: 5480\n",
      "Number of processed documents: 5481\n",
      "Number of processed documents: 5482\n",
      "Number of processed documents: 5483\n",
      "Number of processed documents: 5484\n",
      "Number of processed documents: 5485\n",
      "Number of processed documents: 5486\n",
      "Number of processed documents: 5487\n",
      "Number of processed documents: 5488\n",
      "Number of processed documents: 5489\n",
      "Number of processed documents: 5490\n",
      "Number of processed documents: 5491\n",
      "Number of processed documents: 5492\n",
      "Number of processed documents: 5493\n",
      "Number of processed documents: 5494\n",
      "Number of processed documents: 5495\n",
      "Number of processed documents: 5496\n",
      "Number of processed documents: 5497\n",
      "Write to ../../data/sentiment/norec_sentence/train_pred.docbin...done\n",
      "Number of processed documents: 1\n",
      "Number of processed documents: 2\n",
      "Number of processed documents: 3\n",
      "Number of processed documents: 4\n",
      "Number of processed documents: 5\n",
      "Number of processed documents: 6\n",
      "Number of processed documents: 7\n",
      "Number of processed documents: 8\n",
      "Number of processed documents: 9\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 10\n",
      "Number of processed documents: 11\n",
      "Number of processed documents: 12\n",
      "Number of processed documents: 13\n",
      "Number of processed documents: 14\n",
      "Number of processed documents: 15\n",
      "Number of processed documents: 16\n",
      "Number of processed documents: 17\n",
      "Number of processed documents: 18\n",
      "Number of processed documents: 19\n",
      "Number of processed documents: 20\n",
      "Number of processed documents: 21\n",
      "Number of processed documents: 22\n",
      "Number of processed documents: 23\n",
      "Number of processed documents: 24\n",
      "Number of processed documents: 25\n",
      "Number of processed documents: 26\n",
      "Number of processed documents: 27\n",
      "Number of processed documents: 28\n",
      "Number of processed documents: 29\n",
      "Number of processed documents: 30\n",
      "Number of processed documents: 31\n",
      "Number of processed documents: 32\n",
      "Number of processed documents: 33\n",
      "Number of processed documents: 34\n",
      "Number of processed documents: 35\n",
      "Number of processed documents: 36\n",
      "Number of processed documents: 37\n",
      "Number of processed documents: 38\n",
      "Number of processed documents: 39\n",
      "Number of processed documents: 40\n",
      "Number of processed documents: 41\n",
      "Number of processed documents: 42\n",
      "Number of processed documents: 43\n",
      "Number of processed documents: 44\n",
      "Number of processed documents: 45\n",
      "Number of processed documents: 46\n",
      "Number of processed documents: 47\n",
      "Number of processed documents: 48\n",
      "Number of processed documents: 49\n",
      "Number of processed documents: 50\n",
      "Number of processed documents: 51\n",
      "Number of processed documents: 52\n",
      "Number of processed documents: 53\n",
      "Number of processed documents: 54\n",
      "Number of processed documents: 55\n",
      "Number of processed documents: 56\n",
      "Number of processed documents: 57\n",
      "Number of processed documents: 58\n",
      "Number of processed documents: 59\n",
      "Number of processed documents: 60\n",
      "Number of processed documents: 61\n",
      "Number of processed documents: 62\n",
      "Number of processed documents: 63\n",
      "Number of processed documents: 64\n",
      "Number of processed documents: 65\n",
      "Number of processed documents: 66\n",
      "Number of processed documents: 67\n",
      "Number of processed documents: 68\n",
      "Number of processed documents: 69\n",
      "Number of processed documents: 70\n",
      "Number of processed documents: 71\n",
      "Number of processed documents: 72\n",
      "Number of processed documents: 73\n",
      "Number of processed documents: 74\n",
      "Number of processed documents: 75\n",
      "Number of processed documents: 76\n",
      "Number of processed documents: 77\n",
      "Number of processed documents: 78\n",
      "Number of processed documents: 79\n",
      "Number of processed documents: 80\n",
      "Number of processed documents: 81\n",
      "Number of processed documents: 82\n",
      "Number of processed documents: 83\n",
      "Number of processed documents: 84\n",
      "Number of processed documents: 85\n",
      "Number of processed documents: 86\n",
      "Number of processed documents: 87\n",
      "Number of processed documents: 88\n",
      "Number of processed documents: 89\n",
      "Number of processed documents: 90\n",
      "Number of processed documents: 91\n",
      "Number of processed documents: 92\n",
      "Number of processed documents: 93\n",
      "Number of processed documents: 94\n",
      "Number of processed documents: 95\n",
      "Number of processed documents: 96\n",
      "Number of processed documents: 97\n",
      "Number of processed documents: 98\n",
      "Number of processed documents: 99\n",
      "Number of processed documents: 100\n",
      "Number of processed documents: 101\n",
      "Number of processed documents: 102\n",
      "Number of processed documents: 103\n",
      "Number of processed documents: 104\n",
      "Number of processed documents: 105\n",
      "Number of processed documents: 106\n",
      "Number of processed documents: 107\n",
      "Number of processed documents: 108\n",
      "Number of processed documents: 109\n",
      "Number of processed documents: 110\n",
      "Number of processed documents: 111\n",
      "Number of processed documents: 112\n",
      "Number of processed documents: 113\n",
      "Number of processed documents: 114\n",
      "Number of processed documents: 115\n",
      "Number of processed documents: 116\n",
      "Number of processed documents: 117\n",
      "Number of processed documents: 118\n",
      "Number of processed documents: 119\n",
      "Number of processed documents: 120\n",
      "Number of processed documents: 121\n",
      "Number of processed documents: 122\n",
      "Number of processed documents: 123\n",
      "Number of processed documents: 124\n",
      "Number of processed documents: 125\n",
      "Number of processed documents: 126\n",
      "Number of processed documents: 127\n",
      "Number of processed documents: 128\n",
      "Number of processed documents: 129\n",
      "Number of processed documents: 130\n",
      "Number of processed documents: 131\n",
      "Number of processed documents: 132\n",
      "Number of processed documents: 133\n",
      "Number of processed documents: 134\n",
      "Number of processed documents: 135\n",
      "Number of processed documents: 136\n",
      "Number of processed documents: 137\n",
      "Number of processed documents: 138\n",
      "Number of processed documents: 139\n",
      "Number of processed documents: 140\n",
      "Number of processed documents: 141\n",
      "Number of processed documents: 142\n",
      "Number of processed documents: 143\n",
      "Number of processed documents: 144\n",
      "Number of processed documents: 145\n",
      "Number of processed documents: 146\n",
      "Number of processed documents: 147\n",
      "Number of processed documents: 148\n",
      "Number of processed documents: 149\n",
      "Number of processed documents: 150\n",
      "Number of processed documents: 151\n",
      "Number of processed documents: 152\n",
      "Number of processed documents: 153\n",
      "Number of processed documents: 154\n",
      "Number of processed documents: 155\n",
      "Number of processed documents: 156\n",
      "Number of processed documents: 157\n",
      "Number of processed documents: 158\n",
      "Number of processed documents: 159\n",
      "Number of processed documents: 160\n",
      "Number of processed documents: 161\n",
      "Number of processed documents: 162\n",
      "Number of processed documents: 163\n",
      "Number of processed documents: 164\n",
      "Number of processed documents: 165\n",
      "Number of processed documents: 166\n",
      "Number of processed documents: 167\n",
      "Number of processed documents: 168\n",
      "Number of processed documents: 169\n",
      "Number of processed documents: 170\n",
      "Number of processed documents: 171\n",
      "Number of processed documents: 172\n",
      "Number of processed documents: 173\n",
      "Number of processed documents: 174\n",
      "Number of processed documents: 175\n",
      "Number of processed documents: 176\n",
      "Number of processed documents: 177\n",
      "Number of processed documents: 178\n",
      "Number of processed documents: 179\n",
      "Number of processed documents: 180\n",
      "Number of processed documents: 181\n",
      "Number of processed documents: 182\n",
      "Number of processed documents: 183\n",
      "Number of processed documents: 184\n",
      "Number of processed documents: 185\n",
      "Number of processed documents: 186\n",
      "Number of processed documents: 187\n",
      "Number of processed documents: 188\n",
      "Number of processed documents: 189\n",
      "Number of processed documents: 190\n",
      "Number of processed documents: 191\n",
      "Number of processed documents: 192\n",
      "Number of processed documents: 193\n",
      "Number of processed documents: 194\n",
      "Number of processed documents: 195\n",
      "Number of processed documents: 196\n",
      "Number of processed documents: 197\n",
      "Number of processed documents: 198\n",
      "Number of processed documents: 199\n",
      "Number of processed documents: 200\n",
      "Number of processed documents: 201\n",
      "Number of processed documents: 202\n",
      "Number of processed documents: 203\n",
      "Number of processed documents: 204\n",
      "Number of processed documents: 205\n",
      "Number of processed documents: 206\n",
      "Number of processed documents: 207\n",
      "Number of processed documents: 208\n",
      "Number of processed documents: 209\n",
      "Number of processed documents: 210\n",
      "Number of processed documents: 211\n",
      "Number of processed documents: 212\n",
      "Number of processed documents: 213\n",
      "Number of processed documents: 214\n",
      "Number of processed documents: 215\n",
      "Number of processed documents: 216\n",
      "Number of processed documents: 217\n",
      "Number of processed documents: 218\n",
      "Number of processed documents: 219\n",
      "Number of processed documents: 220\n",
      "Number of processed documents: 221\n",
      "Number of processed documents: 222\n",
      "Number of processed documents: 223\n",
      "Number of processed documents: 224\n",
      "Number of processed documents: 225\n",
      "Number of processed documents: 226\n",
      "Number of processed documents: 227\n",
      "Number of processed documents: 228\n",
      "Number of processed documents: 229\n",
      "Number of processed documents: 230\n",
      "Number of processed documents: 231\n",
      "Number of processed documents: 232\n",
      "Number of processed documents: 233\n",
      "Number of processed documents: 234\n",
      "Number of processed documents: 235\n",
      "Number of processed documents: 236\n",
      "Number of processed documents: 237\n",
      "Number of processed documents: 238\n",
      "Number of processed documents: 239\n",
      "Number of processed documents: 240\n",
      "Number of processed documents: 241\n",
      "Number of processed documents: 242\n",
      "Number of processed documents: 243\n",
      "Number of processed documents: 244\n",
      "Number of processed documents: 245\n",
      "Number of processed documents: 246\n",
      "Number of processed documents: 247\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 248\n",
      "Number of processed documents: 249\n",
      "Number of processed documents: 250\n",
      "Number of processed documents: 251\n",
      "Number of processed documents: 252\n",
      "Number of processed documents: 253\n",
      "Number of processed documents: 254\n",
      "Number of processed documents: 255\n",
      "Number of processed documents: 256\n",
      "Number of processed documents: 257\n",
      "Number of processed documents: 258\n",
      "Number of processed documents: 259\n",
      "Number of processed documents: 260\n",
      "Number of processed documents: 261\n",
      "Number of processed documents: 262\n",
      "Number of processed documents: 263\n",
      "Number of processed documents: 264\n",
      "Number of processed documents: 265\n",
      "Number of processed documents: 266\n",
      "Number of processed documents: 267\n",
      "Number of processed documents: 268\n",
      "Number of processed documents: 269\n",
      "Number of processed documents: 270\n",
      "Number of processed documents: 271\n",
      "Number of processed documents: 272\n",
      "Number of processed documents: 273\n",
      "Number of processed documents: 274\n",
      "Number of processed documents: 275\n",
      "Number of processed documents: 276\n",
      "Number of processed documents: 277\n",
      "Number of processed documents: 278\n",
      "Number of processed documents: 279\n",
      "Number of processed documents: 280\n",
      "Number of processed documents: 281\n",
      "Number of processed documents: 282\n",
      "Number of processed documents: 283\n",
      "Number of processed documents: 284\n",
      "Number of processed documents: 285\n",
      "Number of processed documents: 286\n",
      "Number of processed documents: 287\n",
      "Number of processed documents: 288\n",
      "Number of processed documents: 289\n",
      "Number of processed documents: 290\n",
      "Number of processed documents: 291\n",
      "Number of processed documents: 292\n",
      "Number of processed documents: 293\n",
      "Number of processed documents: 294\n",
      "Number of processed documents: 295\n",
      "Number of processed documents: 296\n",
      "Number of processed documents: 297\n",
      "Number of processed documents: 298\n",
      "Number of processed documents: 299\n",
      "Number of processed documents: 300\n",
      "Number of processed documents: 301\n",
      "Number of processed documents: 302\n",
      "Number of processed documents: 303\n",
      "Number of processed documents: 304\n",
      "Number of processed documents: 305\n",
      "Number of processed documents: 306\n",
      "Number of processed documents: 307\n",
      "Number of processed documents: 308\n",
      "Number of processed documents: 309\n",
      "Number of processed documents: 310\n",
      "Number of processed documents: 311\n",
      "Number of processed documents: 312\n",
      "Number of processed documents: 313\n",
      "Number of processed documents: 314\n",
      "Number of processed documents: 315\n",
      "Number of processed documents: 316\n",
      "Number of processed documents: 317\n",
      "Number of processed documents: 318\n",
      "Number of processed documents: 319\n",
      "Number of processed documents: 320\n",
      "Number of processed documents: 321\n",
      "Number of processed documents: 322\n",
      "Number of processed documents: 323\n",
      "Number of processed documents: 324\n",
      "Number of processed documents: 325\n",
      "Number of processed documents: 326\n",
      "Number of processed documents: 327\n",
      "Number of processed documents: 328\n",
      "Number of processed documents: 329\n",
      "Number of processed documents: 330\n",
      "Number of processed documents: 331\n",
      "Number of processed documents: 332\n",
      "Number of processed documents: 333\n",
      "Number of processed documents: 334\n",
      "Number of processed documents: 335\n",
      "Number of processed documents: 336\n",
      "Number of processed documents: 337\n",
      "Number of processed documents: 338\n",
      "Number of processed documents: 339\n",
      "Number of processed documents: 340\n",
      "Number of processed documents: 341\n",
      "Number of processed documents: 342\n",
      "Number of processed documents: 343\n",
      "Number of processed documents: 344\n",
      "Number of processed documents: 345\n",
      "Number of processed documents: 346\n",
      "Number of processed documents: 347\n",
      "Number of processed documents: 348\n",
      "Number of processed documents: 349\n",
      "Number of processed documents: 350\n",
      "Number of processed documents: 351\n",
      "Number of processed documents: 352\n",
      "Number of processed documents: 353\n",
      "Number of processed documents: 354\n",
      "Number of processed documents: 355\n",
      "Number of processed documents: 356\n",
      "Number of processed documents: 357\n",
      "Number of processed documents: 358\n",
      "Number of processed documents: 359\n",
      "Number of processed documents: 360\n",
      "Number of processed documents: 361\n",
      "Number of processed documents: 362\n",
      "Number of processed documents: 363\n",
      "Number of processed documents: 364\n",
      "Number of processed documents: 365\n",
      "Number of processed documents: 366\n",
      "Number of processed documents: 367\n",
      "Number of processed documents: 368\n",
      "Number of processed documents: 369\n",
      "Number of processed documents: 370\n",
      "Number of processed documents: 371\n",
      "Number of processed documents: 372\n",
      "Number of processed documents: 373\n",
      "Number of processed documents: 374\n",
      "Number of processed documents: 375\n",
      "Number of processed documents: 376\n",
      "Number of processed documents: 377\n",
      "Number of processed documents: 378\n",
      "Number of processed documents: 379\n",
      "Number of processed documents: 380\n",
      "Number of processed documents: 381\n",
      "Number of processed documents: 382\n",
      "Number of processed documents: 383\n",
      "Number of processed documents: 384\n",
      "Number of processed documents: 385\n",
      "Number of processed documents: 386\n",
      "Number of processed documents: 387\n",
      "Number of processed documents: 388\n",
      "Number of processed documents: 389\n",
      "Number of processed documents: 390\n",
      "Number of processed documents: 391\n",
      "Number of processed documents: 392\n",
      "Number of processed documents: 393\n",
      "Number of processed documents: 394\n",
      "Number of processed documents: 395\n",
      "Number of processed documents: 396\n",
      "Number of processed documents: 397\n",
      "Number of processed documents: 398\n",
      "Number of processed documents: 399\n",
      "Number of processed documents: 400\n",
      "Number of processed documents: 401\n",
      "Number of processed documents: 402\n",
      "Number of processed documents: 403\n",
      "Number of processed documents: 404\n",
      "Number of processed documents: 405\n",
      "Number of processed documents: 406\n",
      "Number of processed documents: 407\n",
      "Number of processed documents: 408\n",
      "Number of processed documents: 409\n",
      "Number of processed documents: 410\n",
      "Number of processed documents: 411\n",
      "Number of processed documents: 412\n",
      "Number of processed documents: 413\n",
      "Number of processed documents: 414\n",
      "Number of processed documents: 415\n",
      "Number of processed documents: 416\n",
      "Number of processed documents: 417\n",
      "Number of processed documents: 418\n",
      "Number of processed documents: 419\n",
      "Number of processed documents: 420\n",
      "Number of processed documents: 421\n",
      "Number of processed documents: 422\n",
      "Number of processed documents: 423\n",
      "Number of processed documents: 424\n",
      "Number of processed documents: 425\n",
      "Number of processed documents: 426\n",
      "Number of processed documents: 427\n",
      "Number of processed documents: 428\n",
      "Number of processed documents: 429\n",
      "Number of processed documents: 430\n",
      "Number of processed documents: 431\n",
      "Number of processed documents: 432\n",
      "Number of processed documents: 433\n",
      "Number of processed documents: 434\n",
      "Number of processed documents: 435\n",
      "Number of processed documents: 436\n",
      "Number of processed documents: 437\n",
      "Number of processed documents: 438\n",
      "Number of processed documents: 439\n",
      "Number of processed documents: 440\n",
      "Number of processed documents: 441\n",
      "Number of processed documents: 442\n",
      "Number of processed documents: 443\n",
      "Number of processed documents: 444\n",
      "Number of processed documents: 445\n",
      "Number of processed documents: 446\n",
      "Number of processed documents: 447\n",
      "Number of processed documents: 448\n",
      "Number of processed documents: 449\n",
      "Number of processed documents: 450\n",
      "Number of processed documents: 451\n",
      "Number of processed documents: 452\n",
      "Number of processed documents: 453\n",
      "Number of processed documents: 454\n",
      "Number of processed documents: 455\n",
      "Number of processed documents: 456\n",
      "Number of processed documents: 457\n",
      "Number of processed documents: 458\n",
      "Number of processed documents: 459\n",
      "Number of processed documents: 460\n",
      "Number of processed documents: 461\n",
      "Number of processed documents: 462\n",
      "Number of processed documents: 463\n",
      "Number of processed documents: 464\n",
      "Number of processed documents: 465\n",
      "Number of processed documents: 466\n",
      "Number of processed documents: 467\n",
      "Number of processed documents: 468\n",
      "Number of processed documents: 469\n",
      "Number of processed documents: 470\n",
      "Number of processed documents: 471\n",
      "Number of processed documents: 472\n",
      "Number of processed documents: 473\n",
      "Number of processed documents: 474\n",
      "Number of processed documents: 475\n",
      "Number of processed documents: 476\n",
      "Number of processed documents: 477\n",
      "Number of processed documents: 478\n",
      "Number of processed documents: 479\n",
      "Number of processed documents: 480\n",
      "Number of processed documents: 481\n",
      "Number of processed documents: 482\n",
      "Number of processed documents: 483\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 484\n",
      "Number of processed documents: 485\n",
      "Number of processed documents: 486\n",
      "Number of processed documents: 487\n",
      "Number of processed documents: 488\n",
      "Number of processed documents: 489\n",
      "Number of processed documents: 490\n",
      "Number of processed documents: 491\n",
      "Number of processed documents: 492\n",
      "Number of processed documents: 493\n",
      "Number of processed documents: 494\n",
      "Number of processed documents: 495\n",
      "Number of processed documents: 496\n",
      "Number of processed documents: 497\n",
      "Number of processed documents: 498\n",
      "Number of processed documents: 499\n",
      "Number of processed documents: 500\n",
      "Number of processed documents: 501\n",
      "Number of processed documents: 502\n",
      "Number of processed documents: 503\n",
      "Number of processed documents: 504\n",
      "Number of processed documents: 505\n",
      "Number of processed documents: 506\n",
      "Number of processed documents: 507\n",
      "Number of processed documents: 508\n",
      "Number of processed documents: 509\n",
      "Number of processed documents: 510\n",
      "Number of processed documents: 511\n",
      "Number of processed documents: 512\n",
      "Number of processed documents: 513\n",
      "Number of processed documents: 514\n",
      "Number of processed documents: 515\n",
      "Number of processed documents: 516\n",
      "Number of processed documents: 517\n",
      "Number of processed documents: 518\n",
      "Number of processed documents: 519\n",
      "Number of processed documents: 520\n",
      "Number of processed documents: 521\n",
      "Number of processed documents: 522\n",
      "Number of processed documents: 523\n",
      "Number of processed documents: 524\n",
      "Number of processed documents: 525\n",
      "Number of processed documents: 526\n",
      "Number of processed documents: 527\n",
      "Number of processed documents: 528\n",
      "Number of processed documents: 529\n",
      "Number of processed documents: 530\n",
      "Number of processed documents: 531\n",
      "Number of processed documents: 532\n",
      "Number of processed documents: 533\n",
      "Number of processed documents: 534\n",
      "Number of processed documents: 535\n",
      "Number of processed documents: 536\n",
      "Number of processed documents: 537\n",
      "Number of processed documents: 538\n",
      "Number of processed documents: 539\n",
      "Number of processed documents: 540\n",
      "Number of processed documents: 541\n",
      "Number of processed documents: 542\n",
      "Number of processed documents: 543\n",
      "Number of processed documents: 544\n",
      "Number of processed documents: 545\n",
      "Number of processed documents: 546\n",
      "Number of processed documents: 547\n",
      "Number of processed documents: 548\n",
      "Number of processed documents: 549\n",
      "Number of processed documents: 550\n",
      "Number of processed documents: 551\n",
      "Number of processed documents: 552\n",
      "Number of processed documents: 553\n",
      "Number of processed documents: 554\n",
      "Number of processed documents: 555\n",
      "Number of processed documents: 556\n",
      "Number of processed documents: 557\n",
      "Number of processed documents: 558\n",
      "Number of processed documents: 559\n",
      "Number of processed documents: 560\n",
      "Number of processed documents: 561\n",
      "Number of processed documents: 562\n",
      "Number of processed documents: 563\n",
      "Number of processed documents: 564\n",
      "Number of processed documents: 565\n",
      "Number of processed documents: 566\n",
      "Number of processed documents: 567\n",
      "Number of processed documents: 568\n",
      "Number of processed documents: 569\n",
      "Number of processed documents: 570\n",
      "Number of processed documents: 571\n",
      "Number of processed documents: 572\n",
      "Number of processed documents: 573\n",
      "Number of processed documents: 574\n",
      "Number of processed documents: 575\n",
      "Number of processed documents: 576\n",
      "Number of processed documents: 577\n",
      "Number of processed documents: 578\n",
      "Number of processed documents: 579\n",
      "Number of processed documents: 580\n",
      "Number of processed documents: 581\n",
      "Number of processed documents: 582\n",
      "Number of processed documents: 583\n",
      "Number of processed documents: 584\n",
      "Number of processed documents: 585\n",
      "Number of processed documents: 586\n",
      "Number of processed documents: 587\n",
      "Number of processed documents: 588\n",
      "Number of processed documents: 589\n",
      "Number of processed documents: 590\n",
      "Number of processed documents: 591\n",
      "Number of processed documents: 592\n",
      "Number of processed documents: 593\n",
      "Number of processed documents: 594\n",
      "Number of processed documents: 595\n",
      "Number of processed documents: 596\n",
      "Number of processed documents: 597\n",
      "Number of processed documents: 598\n",
      "Number of processed documents: 599\n",
      "Number of processed documents: 600\n",
      "Number of processed documents: 601\n",
      "Number of processed documents: 602\n",
      "Number of processed documents: 603\n",
      "Number of processed documents: 604\n",
      "Number of processed documents: 605\n",
      "Number of processed documents: 606\n",
      "Number of processed documents: 607\n",
      "Number of processed documents: 608\n",
      "Number of processed documents: 609\n",
      "Number of processed documents: 610\n",
      "Number of processed documents: 611\n",
      "Number of processed documents: 612\n",
      "Number of processed documents: 613\n",
      "Number of processed documents: 614\n",
      "Number of processed documents: 615\n",
      "Number of processed documents: 616\n",
      "Number of processed documents: 617\n",
      "Number of processed documents: 618\n",
      "Number of processed documents: 619\n",
      "Number of processed documents: 620\n",
      "Number of processed documents: 621\n",
      "Number of processed documents: 622\n",
      "Number of processed documents: 623\n",
      "Number of processed documents: 624\n",
      "Number of processed documents: 625\n",
      "Number of processed documents: 626\n",
      "Number of processed documents: 627\n",
      "Number of processed documents: 628\n",
      "Number of processed documents: 629\n",
      "Number of processed documents: 630\n",
      "Number of processed documents: 631\n",
      "Number of processed documents: 632\n",
      "Number of processed documents: 633\n",
      "Number of processed documents: 634\n",
      "Number of processed documents: 635\n",
      "Number of processed documents: 636\n",
      "Number of processed documents: 637\n",
      "Number of processed documents: 638\n",
      "Number of processed documents: 639\n",
      "Number of processed documents: 640\n",
      "Number of processed documents: 641\n",
      "Number of processed documents: 642\n",
      "Number of processed documents: 643\n",
      "Number of processed documents: 644\n",
      "Number of processed documents: 645\n",
      "Number of processed documents: 646\n",
      "Number of processed documents: 647\n",
      "Number of processed documents: 648\n",
      "Number of processed documents: 649\n",
      "Number of processed documents: 650\n",
      "Number of processed documents: 651\n",
      "Number of processed documents: 652\n",
      "Number of processed documents: 653\n",
      "Number of processed documents: 654\n",
      "Number of processed documents: 655\n",
      "Number of processed documents: 656\n",
      "Number of processed documents: 657\n",
      "Number of processed documents: 658\n",
      "Number of processed documents: 659\n",
      "Number of processed documents: 660\n",
      "Number of processed documents: 661\n",
      "Number of processed documents: 662\n",
      "Number of processed documents: 663\n",
      "Number of processed documents: 664\n",
      "Number of processed documents: 665\n",
      "Number of processed documents: 666\n",
      "Number of processed documents: 667\n",
      "Number of processed documents: 668\n",
      "Number of processed documents: 669\n",
      "Number of processed documents: 670\n",
      "Number of processed documents: 671\n",
      "Number of processed documents: 672\n",
      "Number of processed documents: 673\n",
      "Number of processed documents: 674\n",
      "Number of processed documents: 675\n",
      "Number of processed documents: 676\n",
      "Number of processed documents: 677\n",
      "Number of processed documents: 678\n",
      "Number of processed documents: 679\n",
      "Number of processed documents: 680\n",
      "Number of processed documents: 681\n",
      "Number of processed documents: 682\n",
      "Number of processed documents: 683\n",
      "Number of processed documents: 684\n",
      "Number of processed documents: 685\n",
      "Number of processed documents: 686\n",
      "Number of processed documents: 687\n",
      "Number of processed documents: 688\n",
      "Number of processed documents: 689\n",
      "Number of processed documents: 690\n",
      "Number of processed documents: 691\n",
      "Number of processed documents: 692\n",
      "Number of processed documents: 693\n",
      "Number of processed documents: 694\n",
      "Number of processed documents: 695\n",
      "Number of processed documents: 696\n",
      "Number of processed documents: 697\n",
      "Number of processed documents: 698\n",
      "Number of processed documents: 699\n",
      "Number of processed documents: 700\n",
      "Number of processed documents: 701\n",
      "Number of processed documents: 702\n",
      "Number of processed documents: 703\n",
      "Number of processed documents: 704\n",
      "Number of processed documents: 705\n",
      "Number of processed documents: 706\n",
      "Number of processed documents: 707\n",
      "Number of processed documents: 708\n",
      "Number of processed documents: 709\n",
      "Number of processed documents: 710\n",
      "Number of processed documents: 711\n",
      "Number of processed documents: 712\n",
      "Number of processed documents: 713\n",
      "Number of processed documents: 714\n",
      "Number of processed documents: 715\n",
      "Number of processed documents: 716\n",
      "Number of processed documents: 717\n",
      "Number of processed documents: 718\n",
      "Number of processed documents: 719\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 720\n",
      "Number of processed documents: 721\n",
      "Number of processed documents: 722\n",
      "Number of processed documents: 723\n",
      "Number of processed documents: 724\n",
      "Number of processed documents: 725\n",
      "Number of processed documents: 726\n",
      "Number of processed documents: 727\n",
      "Number of processed documents: 728\n",
      "Number of processed documents: 729\n",
      "Number of processed documents: 730\n",
      "Number of processed documents: 731\n",
      "Number of processed documents: 732\n",
      "Number of processed documents: 733\n",
      "Number of processed documents: 734\n",
      "Number of processed documents: 735\n",
      "Number of processed documents: 736\n",
      "Number of processed documents: 737\n",
      "Number of processed documents: 738\n",
      "Number of processed documents: 739\n",
      "Number of processed documents: 740\n",
      "Number of processed documents: 741\n",
      "Number of processed documents: 742\n",
      "Number of processed documents: 743\n",
      "Number of processed documents: 744\n",
      "Number of processed documents: 745\n",
      "Number of processed documents: 746\n",
      "Number of processed documents: 747\n",
      "Number of processed documents: 748\n",
      "Number of processed documents: 749\n",
      "Number of processed documents: 750\n",
      "Number of processed documents: 751\n",
      "Number of processed documents: 752\n",
      "Number of processed documents: 753\n",
      "Number of processed documents: 754\n",
      "Number of processed documents: 755\n",
      "Number of processed documents: 756\n",
      "Number of processed documents: 757\n",
      "Number of processed documents: 758\n",
      "Number of processed documents: 759\n",
      "Number of processed documents: 760\n",
      "Number of processed documents: 761\n",
      "Number of processed documents: 762\n",
      "Number of processed documents: 763\n",
      "Number of processed documents: 764\n",
      "Number of processed documents: 765\n",
      "Number of processed documents: 766\n",
      "Number of processed documents: 767\n",
      "Number of processed documents: 768\n",
      "Number of processed documents: 769\n",
      "Number of processed documents: 770\n",
      "Number of processed documents: 771\n",
      "Number of processed documents: 772\n",
      "Number of processed documents: 773\n",
      "Number of processed documents: 774\n",
      "Number of processed documents: 775\n",
      "Number of processed documents: 776\n",
      "Number of processed documents: 777\n",
      "Number of processed documents: 778\n",
      "Number of processed documents: 779\n",
      "Number of processed documents: 780\n",
      "Number of processed documents: 781\n",
      "Number of processed documents: 782\n",
      "Number of processed documents: 783\n",
      "Number of processed documents: 784\n",
      "Number of processed documents: 785\n",
      "Number of processed documents: 786\n",
      "Number of processed documents: 787\n",
      "Number of processed documents: 788\n",
      "Number of processed documents: 789\n",
      "Number of processed documents: 790\n",
      "Number of processed documents: 791\n",
      "Number of processed documents: 792\n",
      "Number of processed documents: 793\n",
      "Number of processed documents: 794\n",
      "Number of processed documents: 795\n",
      "Number of processed documents: 796\n",
      "Number of processed documents: 797\n",
      "Number of processed documents: 798\n",
      "Number of processed documents: 799\n",
      "Number of processed documents: 800\n",
      "Number of processed documents: 801\n",
      "Number of processed documents: 802\n",
      "Number of processed documents: 803\n",
      "Number of processed documents: 804\n",
      "Number of processed documents: 805\n",
      "Number of processed documents: 806\n",
      "Number of processed documents: 807\n",
      "Number of processed documents: 808\n",
      "Number of processed documents: 809\n",
      "Number of processed documents: 810\n",
      "Number of processed documents: 811\n",
      "Number of processed documents: 812\n",
      "Number of processed documents: 813\n",
      "Number of processed documents: 814\n",
      "Number of processed documents: 815\n",
      "Number of processed documents: 816\n",
      "Number of processed documents: 817\n",
      "Number of processed documents: 818\n",
      "Number of processed documents: 819\n",
      "Number of processed documents: 820\n",
      "Number of processed documents: 821\n",
      "Number of processed documents: 822\n",
      "Number of processed documents: 823\n",
      "Number of processed documents: 824\n",
      "Number of processed documents: 825\n",
      "Number of processed documents: 826\n",
      "Number of processed documents: 827\n",
      "Number of processed documents: 828\n",
      "Number of processed documents: 829\n",
      "Number of processed documents: 830\n",
      "Number of processed documents: 831\n",
      "Number of processed documents: 832\n",
      "Number of processed documents: 833\n",
      "Number of processed documents: 834\n",
      "Number of processed documents: 835\n",
      "Number of processed documents: 836\n",
      "Number of processed documents: 837\n",
      "Number of processed documents: 838\n",
      "Number of processed documents: 839\n",
      "Number of processed documents: 840\n",
      "Number of processed documents: 841\n",
      "Number of processed documents: 842\n",
      "Number of processed documents: 843\n",
      "Number of processed documents: 844\n",
      "Number of processed documents: 845\n",
      "Number of processed documents: 846\n",
      "Number of processed documents: 847\n",
      "Number of processed documents: 848\n",
      "Number of processed documents: 849\n",
      "Number of processed documents: 850\n",
      "Number of processed documents: 851\n",
      "Number of processed documents: 852\n",
      "Number of processed documents: 853\n",
      "Number of processed documents: 854\n",
      "Number of processed documents: 855\n",
      "Number of processed documents: 856\n",
      "Number of processed documents: 857\n",
      "Number of processed documents: 858\n",
      "Number of processed documents: 859\n",
      "Number of processed documents: 860\n",
      "Number of processed documents: 861\n",
      "Number of processed documents: 862\n",
      "Number of processed documents: 863\n",
      "Number of processed documents: 864\n",
      "Number of processed documents: 865\n",
      "Number of processed documents: 866\n",
      "Number of processed documents: 867\n",
      "Number of processed documents: 868\n",
      "Number of processed documents: 869\n",
      "Number of processed documents: 870\n",
      "Number of processed documents: 871\n",
      "Number of processed documents: 872\n",
      "Number of processed documents: 873\n",
      "Number of processed documents: 874\n",
      "Number of processed documents: 875\n",
      "Number of processed documents: 876\n",
      "Number of processed documents: 877\n",
      "Number of processed documents: 878\n",
      "Number of processed documents: 879\n",
      "Number of processed documents: 880\n",
      "Number of processed documents: 881\n",
      "Number of processed documents: 882\n",
      "Number of processed documents: 883\n",
      "Number of processed documents: 884\n",
      "Number of processed documents: 885\n",
      "Number of processed documents: 886\n",
      "Number of processed documents: 887\n",
      "Number of processed documents: 888\n",
      "Number of processed documents: 889\n",
      "Number of processed documents: 890\n",
      "Number of processed documents: 891\n",
      "Number of processed documents: 892\n",
      "Number of processed documents: 893\n",
      "Number of processed documents: 894\n",
      "Number of processed documents: 895\n",
      "Number of processed documents: 896\n",
      "Number of processed documents: 897\n",
      "Number of processed documents: 898\n",
      "Number of processed documents: 899\n",
      "Number of processed documents: 900\n",
      "Number of processed documents: 901\n",
      "Number of processed documents: 902\n",
      "Number of processed documents: 903\n",
      "Number of processed documents: 904\n",
      "Number of processed documents: 905\n",
      "Number of processed documents: 906\n",
      "Number of processed documents: 907\n",
      "Number of processed documents: 908\n",
      "Number of processed documents: 909\n",
      "Number of processed documents: 910\n",
      "Number of processed documents: 911\n",
      "Number of processed documents: 912\n",
      "Number of processed documents: 913\n",
      "Number of processed documents: 914\n",
      "Number of processed documents: 915\n",
      "Number of processed documents: 916\n",
      "Number of processed documents: 917\n",
      "Number of processed documents: 918\n",
      "Number of processed documents: 919\n",
      "Number of processed documents: 920\n",
      "Number of processed documents: 921\n",
      "Number of processed documents: 922\n",
      "Number of processed documents: 923\n",
      "Number of processed documents: 924\n",
      "Number of processed documents: 925\n",
      "Number of processed documents: 926\n",
      "Number of processed documents: 927\n",
      "Number of processed documents: 928\n",
      "Number of processed documents: 929\n",
      "Number of processed documents: 930\n",
      "Number of processed documents: 931\n",
      "Number of processed documents: 932\n",
      "Number of processed documents: 933\n",
      "Number of processed documents: 934\n",
      "Number of processed documents: 935\n",
      "Number of processed documents: 936\n",
      "Number of processed documents: 937\n",
      "Number of processed documents: 938\n",
      "Number of processed documents: 939\n",
      "Number of processed documents: 940\n",
      "Number of processed documents: 941\n",
      "Number of processed documents: 942\n",
      "Number of processed documents: 943\n",
      "Number of processed documents: 944\n",
      "Number of processed documents: 945\n",
      "Number of processed documents: 946\n",
      "Number of processed documents: 947\n",
      "Number of processed documents: 948\n",
      "Number of processed documents: 949\n",
      "Number of processed documents: 950\n",
      "Number of processed documents: 951\n",
      "Number of processed documents: 952\n",
      "Number of processed documents: 953\n",
      "Number of processed documents: 954\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 955\n",
      "Number of processed documents: 956\n",
      "Number of processed documents: 957\n",
      "Number of processed documents: 958\n",
      "Number of processed documents: 959\n",
      "Number of processed documents: 960\n",
      "Number of processed documents: 961\n",
      "Number of processed documents: 962\n",
      "Number of processed documents: 963\n",
      "Number of processed documents: 964\n",
      "Number of processed documents: 965\n",
      "Number of processed documents: 966\n",
      "Number of processed documents: 967\n",
      "Number of processed documents: 968\n",
      "Number of processed documents: 969\n",
      "Number of processed documents: 970\n",
      "Number of processed documents: 971\n",
      "Number of processed documents: 972\n",
      "Number of processed documents: 973\n",
      "Number of processed documents: 974\n",
      "Number of processed documents: 975\n",
      "Number of processed documents: 976\n",
      "Number of processed documents: 977\n",
      "Number of processed documents: 978\n",
      "Number of processed documents: 979\n",
      "Number of processed documents: 980\n",
      "Number of processed documents: 981\n",
      "Number of processed documents: 982\n",
      "Number of processed documents: 983\n",
      "Number of processed documents: 984\n",
      "Number of processed documents: 985\n",
      "Number of processed documents: 986\n",
      "Number of processed documents: 987\n",
      "Number of processed documents: 988\n",
      "Number of processed documents: 989\n",
      "Number of processed documents: 990\n",
      "Number of processed documents: 991\n",
      "Number of processed documents: 992\n",
      "Number of processed documents: 993\n",
      "Number of processed documents: 994\n",
      "Number of processed documents: 995\n",
      "Number of processed documents: 996\n",
      "Number of processed documents: 997\n",
      "Number of processed documents: 998\n",
      "Number of processed documents: 999\n",
      "Number of processed documents: 1000\n",
      "Number of processed documents: 1001\n",
      "Number of processed documents: 1002\n",
      "Number of processed documents: 1003\n",
      "Number of processed documents: 1004\n",
      "Number of processed documents: 1005\n",
      "Number of processed documents: 1006\n",
      "Number of processed documents: 1007\n",
      "Number of processed documents: 1008\n",
      "Number of processed documents: 1009\n",
      "Number of processed documents: 1010\n",
      "Number of processed documents: 1011\n",
      "Number of processed documents: 1012\n",
      "Number of processed documents: 1013\n",
      "Number of processed documents: 1014\n",
      "Number of processed documents: 1015\n",
      "Number of processed documents: 1016\n",
      "Number of processed documents: 1017\n",
      "Number of processed documents: 1018\n",
      "Number of processed documents: 1019\n",
      "Number of processed documents: 1020\n",
      "Number of processed documents: 1021\n",
      "Number of processed documents: 1022\n",
      "Number of processed documents: 1023\n",
      "Number of processed documents: 1024\n",
      "Number of processed documents: 1025\n",
      "Number of processed documents: 1026\n",
      "Number of processed documents: 1027\n",
      "Number of processed documents: 1028\n",
      "Number of processed documents: 1029\n",
      "Number of processed documents: 1030\n",
      "Number of processed documents: 1031\n",
      "Number of processed documents: 1032\n",
      "Number of processed documents: 1033\n",
      "Number of processed documents: 1034\n",
      "Number of processed documents: 1035\n",
      "Number of processed documents: 1036\n",
      "Number of processed documents: 1037\n",
      "Number of processed documents: 1038\n",
      "Number of processed documents: 1039\n",
      "Number of processed documents: 1040\n",
      "Number of processed documents: 1041\n",
      "Number of processed documents: 1042\n",
      "Number of processed documents: 1043\n",
      "Number of processed documents: 1044\n",
      "Number of processed documents: 1045\n",
      "Number of processed documents: 1046\n",
      "Number of processed documents: 1047\n",
      "Number of processed documents: 1048\n",
      "Number of processed documents: 1049\n",
      "Number of processed documents: 1050\n",
      "Number of processed documents: 1051\n",
      "Number of processed documents: 1052\n",
      "Number of processed documents: 1053\n",
      "Number of processed documents: 1054\n",
      "Number of processed documents: 1055\n",
      "Number of processed documents: 1056\n",
      "Number of processed documents: 1057\n",
      "Number of processed documents: 1058\n",
      "Number of processed documents: 1059\n",
      "Number of processed documents: 1060\n",
      "Number of processed documents: 1061\n",
      "Number of processed documents: 1062\n",
      "Number of processed documents: 1063\n",
      "Number of processed documents: 1064\n",
      "Write to ../../data/sentiment/norec_sentence/dev_pred.docbin...done\n",
      "Number of processed documents: 1\n",
      "Number of processed documents: 2\n",
      "Number of processed documents: 3\n",
      "Number of processed documents: 4\n",
      "Number of processed documents: 5\n",
      "Number of processed documents: 6\n",
      "Number of processed documents: 7\n",
      "Number of processed documents: 8\n",
      "Number of processed documents: 9\n",
      "Number of processed documents: 10\n",
      "Number of processed documents: 11\n",
      "Number of processed documents: 12\n",
      "Number of processed documents: 13\n",
      "Number of processed documents: 14\n",
      "Number of processed documents: 15\n",
      "Number of processed documents: 16\n",
      "Number of processed documents: 17\n",
      "Number of processed documents: 18\n",
      "Number of processed documents: 19\n",
      "Number of processed documents: 20\n",
      "Number of processed documents: 21\n",
      "Number of processed documents: 22\n",
      "Number of processed documents: 23\n",
      "Number of processed documents: 24\n",
      "Number of processed documents: 25\n",
      "Number of processed documents: 26\n",
      "Number of processed documents: 27\n",
      "Number of processed documents: 28\n",
      "Number of processed documents: 29\n",
      "Number of processed documents: 30\n",
      "Number of processed documents: 31\n",
      "Number of processed documents: 32\n",
      "Number of processed documents: 33\n",
      "Number of processed documents: 34\n",
      "Number of processed documents: 35\n",
      "Number of processed documents: 36\n",
      "Number of processed documents: 37\n",
      "Number of processed documents: 38\n",
      "Number of processed documents: 39\n",
      "Number of processed documents: 40\n",
      "Number of processed documents: 41\n",
      "Number of processed documents: 42\n",
      "Number of processed documents: 43\n",
      "Number of processed documents: 44\n",
      "Number of processed documents: 45\n",
      "Number of processed documents: 46\n",
      "Number of processed documents: 47\n",
      "Number of processed documents: 48\n",
      "Number of processed documents: 49\n",
      "Number of processed documents: 50\n",
      "Number of processed documents: 51\n",
      "Number of processed documents: 52\n",
      "Number of processed documents: 53\n",
      "Number of processed documents: 54\n",
      "Number of processed documents: 55\n",
      "Number of processed documents: 56\n",
      "Number of processed documents: 57\n",
      "Number of processed documents: 58\n",
      "Number of processed documents: 59\n",
      "Number of processed documents: 60\n",
      "Number of processed documents: 61\n",
      "Number of processed documents: 62\n",
      "Number of processed documents: 63\n",
      "Number of processed documents: 64\n",
      "Number of processed documents: 65\n",
      "Number of processed documents: 66\n",
      "Number of processed documents: 67\n",
      "Number of processed documents: 68\n",
      "Number of processed documents: 69\n",
      "Number of processed documents: 70\n",
      "Number of processed documents: 71\n",
      "Number of processed documents: 72\n",
      "Number of processed documents: 73\n",
      "Number of processed documents: 74\n",
      "Number of processed documents: 75\n",
      "Number of processed documents: 76\n",
      "Number of processed documents: 77\n",
      "Number of processed documents: 78\n",
      "Number of processed documents: 79\n",
      "Number of processed documents: 80\n",
      "Number of processed documents: 81\n",
      "Number of processed documents: 82\n",
      "Number of processed documents: 83\n",
      "Number of processed documents: 84\n",
      "Number of processed documents: 85\n",
      "Number of processed documents: 86\n",
      "Number of processed documents: 87\n",
      "Number of processed documents: 88\n",
      "Number of processed documents: 89\n",
      "Number of processed documents: 90\n",
      "Number of processed documents: 91\n",
      "Number of processed documents: 92\n",
      "Number of processed documents: 93\n",
      "Number of processed documents: 94\n",
      "Number of processed documents: 95\n",
      "Number of processed documents: 96\n",
      "Number of processed documents: 97\n",
      "Number of processed documents: 98\n",
      "Number of processed documents: 99\n",
      "Number of processed documents: 100\n",
      "Number of processed documents: 101\n",
      "Number of processed documents: 102\n",
      "Number of processed documents: 103\n",
      "Number of processed documents: 104\n",
      "Number of processed documents: 105\n",
      "Number of processed documents: 106\n",
      "Number of processed documents: 107\n",
      "Number of processed documents: 108\n",
      "Number of processed documents: 109\n",
      "Number of processed documents: 110\n",
      "Number of processed documents: 111\n",
      "Number of processed documents: 112\n",
      "Number of processed documents: 113\n",
      "Number of processed documents: 114\n",
      "Number of processed documents: 115\n",
      "Number of processed documents: 116\n",
      "Number of processed documents: 117\n",
      "Number of processed documents: 118\n",
      "Number of processed documents: 119\n",
      "Number of processed documents: 120\n",
      "Number of processed documents: 121\n",
      "Number of processed documents: 122\n",
      "Number of processed documents: 123\n",
      "Number of processed documents: 124\n",
      "Number of processed documents: 125\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 126\n",
      "Number of processed documents: 127\n",
      "Number of processed documents: 128\n",
      "Number of processed documents: 129\n",
      "Number of processed documents: 130\n",
      "Number of processed documents: 131\n",
      "Number of processed documents: 132\n",
      "Number of processed documents: 133\n",
      "Number of processed documents: 134\n",
      "Number of processed documents: 135\n",
      "Number of processed documents: 136\n",
      "Number of processed documents: 137\n",
      "Number of processed documents: 138\n",
      "Number of processed documents: 139\n",
      "Number of processed documents: 140\n",
      "Number of processed documents: 141\n",
      "Number of processed documents: 142\n",
      "Number of processed documents: 143\n",
      "Number of processed documents: 144\n",
      "Number of processed documents: 145\n",
      "Number of processed documents: 146\n",
      "Number of processed documents: 147\n",
      "Number of processed documents: 148\n",
      "Number of processed documents: 149\n",
      "Number of processed documents: 150\n",
      "Number of processed documents: 151\n",
      "Number of processed documents: 152\n",
      "Number of processed documents: 153\n",
      "Number of processed documents: 154\n",
      "Number of processed documents: 155\n",
      "Number of processed documents: 156\n",
      "Number of processed documents: 157\n",
      "Number of processed documents: 158\n",
      "Number of processed documents: 159\n",
      "Number of processed documents: 160\n",
      "Number of processed documents: 161\n",
      "Number of processed documents: 162\n",
      "Number of processed documents: 163\n",
      "Number of processed documents: 164\n",
      "Number of processed documents: 165\n",
      "Number of processed documents: 166\n",
      "Number of processed documents: 167\n",
      "Number of processed documents: 168\n",
      "Number of processed documents: 169\n",
      "Number of processed documents: 170\n",
      "Number of processed documents: 171\n",
      "Number of processed documents: 172\n",
      "Number of processed documents: 173\n",
      "Number of processed documents: 174\n",
      "Number of processed documents: 175\n",
      "Number of processed documents: 176\n",
      "Number of processed documents: 177\n",
      "Number of processed documents: 178\n",
      "Number of processed documents: 179\n",
      "Number of processed documents: 180\n",
      "Number of processed documents: 181\n",
      "Number of processed documents: 182\n",
      "Number of processed documents: 183\n",
      "Number of processed documents: 184\n",
      "Number of processed documents: 185\n",
      "Number of processed documents: 186\n",
      "Number of processed documents: 187\n",
      "Number of processed documents: 188\n",
      "Number of processed documents: 189\n",
      "Number of processed documents: 190\n",
      "Number of processed documents: 191\n",
      "Number of processed documents: 192\n",
      "Number of processed documents: 193\n",
      "Number of processed documents: 194\n",
      "Number of processed documents: 195\n",
      "Number of processed documents: 196\n",
      "Number of processed documents: 197\n",
      "Number of processed documents: 198\n",
      "Number of processed documents: 199\n",
      "Number of processed documents: 200\n",
      "Number of processed documents: 201\n",
      "Number of processed documents: 202\n",
      "Number of processed documents: 203\n",
      "Number of processed documents: 204\n",
      "Number of processed documents: 205\n",
      "Number of processed documents: 206\n",
      "Number of processed documents: 207\n",
      "Number of processed documents: 208\n",
      "Number of processed documents: 209\n",
      "Number of processed documents: 210\n",
      "Number of processed documents: 211\n",
      "Number of processed documents: 212\n",
      "Number of processed documents: 213\n",
      "Number of processed documents: 214\n",
      "Number of processed documents: 215\n",
      "Number of processed documents: 216\n",
      "Number of processed documents: 217\n",
      "Number of processed documents: 218\n",
      "Number of processed documents: 219\n",
      "Number of processed documents: 220\n",
      "Number of processed documents: 221\n",
      "Number of processed documents: 222\n",
      "Number of processed documents: 223\n",
      "Number of processed documents: 224\n",
      "Number of processed documents: 225\n",
      "Number of processed documents: 226\n",
      "Number of processed documents: 227\n",
      "Number of processed documents: 228\n",
      "Number of processed documents: 229\n",
      "Number of processed documents: 230\n",
      "Number of processed documents: 231\n",
      "Number of processed documents: 232\n",
      "Number of processed documents: 233\n",
      "Number of processed documents: 234\n",
      "Number of processed documents: 235\n",
      "Number of processed documents: 236\n",
      "Number of processed documents: 237\n",
      "Number of processed documents: 238\n",
      "Number of processed documents: 239\n",
      "Number of processed documents: 240\n",
      "Number of processed documents: 241\n",
      "Number of processed documents: 242\n",
      "Number of processed documents: 243\n",
      "Number of processed documents: 244\n",
      "Number of processed documents: 245\n",
      "Number of processed documents: 246\n",
      "Number of processed documents: 247\n",
      "Number of processed documents: 248\n",
      "Number of processed documents: 249\n",
      "Number of processed documents: 250\n",
      "Number of processed documents: 251\n",
      "Number of processed documents: 252\n",
      "Number of processed documents: 253\n",
      "Number of processed documents: 254\n",
      "Number of processed documents: 255\n",
      "Number of processed documents: 256\n",
      "Number of processed documents: 257\n",
      "Number of processed documents: 258\n",
      "Number of processed documents: 259\n",
      "Number of processed documents: 260\n",
      "Number of processed documents: 261\n",
      "Number of processed documents: 262\n",
      "Number of processed documents: 263\n",
      "Number of processed documents: 264\n",
      "Number of processed documents: 265\n",
      "Number of processed documents: 266\n",
      "Number of processed documents: 267\n",
      "Number of processed documents: 268\n",
      "Number of processed documents: 269\n",
      "Number of processed documents: 270\n",
      "Number of processed documents: 271\n",
      "Number of processed documents: 272\n",
      "Number of processed documents: 273\n",
      "Number of processed documents: 274\n",
      "Number of processed documents: 275\n",
      "Number of processed documents: 276\n",
      "Number of processed documents: 277\n",
      "Number of processed documents: 278\n",
      "Number of processed documents: 279\n",
      "Number of processed documents: 280\n",
      "Number of processed documents: 281\n",
      "Number of processed documents: 282\n",
      "Number of processed documents: 283\n",
      "Number of processed documents: 284\n",
      "Number of processed documents: 285\n",
      "Number of processed documents: 286\n",
      "Number of processed documents: 287\n",
      "Number of processed documents: 288\n",
      "Number of processed documents: 289\n",
      "Number of processed documents: 290\n",
      "Number of processed documents: 291\n",
      "Number of processed documents: 292\n",
      "Number of processed documents: 293\n",
      "Number of processed documents: 294\n",
      "Number of processed documents: 295\n",
      "Number of processed documents: 296\n",
      "Number of processed documents: 297\n",
      "Number of processed documents: 298\n",
      "Number of processed documents: 299\n",
      "Number of processed documents: 300\n",
      "Number of processed documents: 301\n",
      "Number of processed documents: 302\n",
      "Number of processed documents: 303\n",
      "Number of processed documents: 304\n",
      "Number of processed documents: 305\n",
      "Number of processed documents: 306\n",
      "Number of processed documents: 307\n",
      "Number of processed documents: 308\n",
      "Number of processed documents: 309\n",
      "Number of processed documents: 310\n",
      "Number of processed documents: 311\n",
      "Number of processed documents: 312\n",
      "Number of processed documents: 313\n",
      "Number of processed documents: 314\n",
      "Number of processed documents: 315\n",
      "Number of processed documents: 316\n",
      "Number of processed documents: 317\n",
      "Number of processed documents: 318\n",
      "Number of processed documents: 319\n",
      "Number of processed documents: 320\n",
      "Number of processed documents: 321\n",
      "Number of processed documents: 322\n",
      "Number of processed documents: 323\n",
      "Number of processed documents: 324\n",
      "Number of processed documents: 325\n",
      "Number of processed documents: 326\n",
      "Number of processed documents: 327\n",
      "Number of processed documents: 328\n",
      "Number of processed documents: 329\n",
      "Number of processed documents: 330\n",
      "Number of processed documents: 331\n",
      "Number of processed documents: 332\n",
      "Number of processed documents: 333\n",
      "Number of processed documents: 334\n",
      "Number of processed documents: 335\n",
      "Number of processed documents: 336\n",
      "Number of processed documents: 337\n",
      "Number of processed documents: 338\n",
      "Number of processed documents: 339\n",
      "Number of processed documents: 340\n",
      "Number of processed documents: 341\n",
      "Number of processed documents: 342\n",
      "Number of processed documents: 343\n",
      "Number of processed documents: 344\n",
      "Number of processed documents: 345\n",
      "Number of processed documents: 346\n",
      "Number of processed documents: 347\n",
      "Number of processed documents: 348\n",
      "Number of processed documents: 349\n",
      "Number of processed documents: 350\n",
      "Number of processed documents: 351\n",
      "Number of processed documents: 352\n",
      "Number of processed documents: 353\n",
      "Number of processed documents: 354\n",
      "Number of processed documents: 355\n",
      "Number of processed documents: 356\n",
      "Number of processed documents: 357\n",
      "Number of processed documents: 358\n",
      "Number of processed documents: 359\n",
      "Number of processed documents: 360\n",
      "Number of processed documents: 361\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 362\n",
      "Number of processed documents: 363\n",
      "Number of processed documents: 364\n",
      "Number of processed documents: 365\n",
      "Number of processed documents: 366\n",
      "Number of processed documents: 367\n",
      "Number of processed documents: 368\n",
      "Number of processed documents: 369\n",
      "Number of processed documents: 370\n",
      "Number of processed documents: 371\n",
      "Number of processed documents: 372\n",
      "Number of processed documents: 373\n",
      "Number of processed documents: 374\n",
      "Number of processed documents: 375\n",
      "Number of processed documents: 376\n",
      "Number of processed documents: 377\n",
      "Number of processed documents: 378\n",
      "Number of processed documents: 379\n",
      "Number of processed documents: 380\n",
      "Number of processed documents: 381\n",
      "Number of processed documents: 382\n",
      "Number of processed documents: 383\n",
      "Number of processed documents: 384\n",
      "Number of processed documents: 385\n",
      "Number of processed documents: 386\n",
      "Number of processed documents: 387\n",
      "Number of processed documents: 388\n",
      "Number of processed documents: 389\n",
      "Number of processed documents: 390\n",
      "Number of processed documents: 391\n",
      "Number of processed documents: 392\n",
      "Number of processed documents: 393\n",
      "Number of processed documents: 394\n",
      "Number of processed documents: 395\n",
      "Number of processed documents: 396\n",
      "Number of processed documents: 397\n",
      "Number of processed documents: 398\n",
      "Number of processed documents: 399\n",
      "Number of processed documents: 400\n",
      "Number of processed documents: 401\n",
      "Number of processed documents: 402\n",
      "Number of processed documents: 403\n",
      "Number of processed documents: 404\n",
      "Number of processed documents: 405\n",
      "Number of processed documents: 406\n",
      "Number of processed documents: 407\n",
      "Number of processed documents: 408\n",
      "Number of processed documents: 409\n",
      "Number of processed documents: 410\n",
      "Number of processed documents: 411\n",
      "Number of processed documents: 412\n",
      "Number of processed documents: 413\n",
      "Number of processed documents: 414\n",
      "Number of processed documents: 415\n",
      "Number of processed documents: 416\n",
      "Number of processed documents: 417\n",
      "Number of processed documents: 418\n",
      "Number of processed documents: 419\n",
      "Number of processed documents: 420\n",
      "Number of processed documents: 421\n",
      "Number of processed documents: 422\n",
      "Number of processed documents: 423\n",
      "Number of processed documents: 424\n",
      "Number of processed documents: 425\n",
      "Number of processed documents: 426\n",
      "Number of processed documents: 427\n",
      "Number of processed documents: 428\n",
      "Number of processed documents: 429\n",
      "Number of processed documents: 430\n",
      "Number of processed documents: 431\n",
      "Number of processed documents: 432\n",
      "Number of processed documents: 433\n",
      "Number of processed documents: 434\n",
      "Number of processed documents: 435\n",
      "Number of processed documents: 436\n",
      "Number of processed documents: 437\n",
      "Number of processed documents: 438\n",
      "Number of processed documents: 439\n",
      "Number of processed documents: 440\n",
      "Number of processed documents: 441\n",
      "Number of processed documents: 442\n",
      "Number of processed documents: 443\n",
      "Number of processed documents: 444\n",
      "Number of processed documents: 445\n",
      "Number of processed documents: 446\n",
      "Number of processed documents: 447\n",
      "Number of processed documents: 448\n",
      "Number of processed documents: 449\n",
      "Number of processed documents: 450\n",
      "Number of processed documents: 451\n",
      "Number of processed documents: 452\n",
      "Number of processed documents: 453\n",
      "Number of processed documents: 454\n",
      "Number of processed documents: 455\n",
      "Number of processed documents: 456\n",
      "Number of processed documents: 457\n",
      "Number of processed documents: 458\n",
      "Number of processed documents: 459\n",
      "Number of processed documents: 460\n",
      "Number of processed documents: 461\n",
      "Number of processed documents: 462\n",
      "Number of processed documents: 463\n",
      "Number of processed documents: 464\n",
      "Number of processed documents: 465\n",
      "Number of processed documents: 466\n",
      "Number of processed documents: 467\n",
      "Number of processed documents: 468\n",
      "Number of processed documents: 469\n",
      "Number of processed documents: 470\n",
      "Number of processed documents: 471\n",
      "Number of processed documents: 472\n",
      "Number of processed documents: 473\n",
      "Number of processed documents: 474\n",
      "Number of processed documents: 475\n",
      "Number of processed documents: 476\n",
      "Number of processed documents: 477\n",
      "Number of processed documents: 478\n",
      "Number of processed documents: 479\n",
      "Number of processed documents: 480\n",
      "Number of processed documents: 481\n",
      "Number of processed documents: 482\n",
      "Number of processed documents: 483\n",
      "Number of processed documents: 484\n",
      "Number of processed documents: 485\n",
      "Number of processed documents: 486\n",
      "Number of processed documents: 487\n",
      "Number of processed documents: 488\n",
      "Number of processed documents: 489\n",
      "Number of processed documents: 490\n",
      "Number of processed documents: 491\n",
      "Number of processed documents: 492\n",
      "Number of processed documents: 493\n",
      "Number of processed documents: 494\n",
      "Number of processed documents: 495\n",
      "Number of processed documents: 496\n",
      "Number of processed documents: 497\n",
      "Number of processed documents: 498\n",
      "Number of processed documents: 499\n",
      "Number of processed documents: 500\n",
      "Number of processed documents: 501\n",
      "Number of processed documents: 502\n",
      "Number of processed documents: 503\n",
      "Number of processed documents: 504\n",
      "Number of processed documents: 505\n",
      "Number of processed documents: 506\n",
      "Number of processed documents: 507\n",
      "Number of processed documents: 508\n",
      "Number of processed documents: 509\n",
      "Number of processed documents: 510\n",
      "Number of processed documents: 511\n",
      "Number of processed documents: 512\n",
      "Number of processed documents: 513\n",
      "Number of processed documents: 514\n",
      "Number of processed documents: 515\n",
      "Number of processed documents: 516\n",
      "Number of processed documents: 517\n",
      "Number of processed documents: 518\n",
      "Number of processed documents: 519\n",
      "Number of processed documents: 520\n",
      "Number of processed documents: 521\n",
      "Number of processed documents: 522\n",
      "Number of processed documents: 523\n",
      "Number of processed documents: 524\n",
      "Number of processed documents: 525\n",
      "Number of processed documents: 526\n",
      "Number of processed documents: 527\n",
      "Number of processed documents: 528\n",
      "Number of processed documents: 529\n",
      "Number of processed documents: 530\n",
      "Number of processed documents: 531\n",
      "Number of processed documents: 532\n",
      "Number of processed documents: 533\n",
      "Number of processed documents: 534\n",
      "Number of processed documents: 535\n",
      "Number of processed documents: 536\n",
      "Number of processed documents: 537\n",
      "Number of processed documents: 538\n",
      "Number of processed documents: 539\n",
      "Number of processed documents: 540\n",
      "Number of processed documents: 541\n",
      "Number of processed documents: 542\n",
      "Number of processed documents: 543\n",
      "Number of processed documents: 544\n",
      "Number of processed documents: 545\n",
      "Number of processed documents: 546\n",
      "Number of processed documents: 547\n",
      "Number of processed documents: 548\n",
      "Number of processed documents: 549\n",
      "Number of processed documents: 550\n",
      "Number of processed documents: 551\n",
      "Number of processed documents: 552\n",
      "Number of processed documents: 553\n",
      "Number of processed documents: 554\n",
      "Number of processed documents: 555\n",
      "Number of processed documents: 556\n",
      "Number of processed documents: 557\n",
      "Number of processed documents: 558\n",
      "Number of processed documents: 559\n",
      "Number of processed documents: 560\n",
      "Number of processed documents: 561\n",
      "Number of processed documents: 562\n",
      "Number of processed documents: 563\n",
      "Number of processed documents: 564\n",
      "Number of processed documents: 565\n",
      "Number of processed documents: 566\n",
      "Number of processed documents: 567\n",
      "Number of processed documents: 568\n",
      "Number of processed documents: 569\n",
      "Number of processed documents: 570\n",
      "Number of processed documents: 571\n",
      "Number of processed documents: 572\n",
      "Number of processed documents: 573\n",
      "Number of processed documents: 574\n",
      "Number of processed documents: 575\n",
      "Number of processed documents: 576\n",
      "Number of processed documents: 577\n",
      "Number of processed documents: 578\n",
      "Number of processed documents: 579\n",
      "Number of processed documents: 580\n",
      "Number of processed documents: 581\n",
      "Number of processed documents: 582\n",
      "Number of processed documents: 583\n",
      "Number of processed documents: 584\n",
      "Number of processed documents: 585\n",
      "Number of processed documents: 586\n",
      "Number of processed documents: 587\n",
      "Number of processed documents: 588\n",
      "Number of processed documents: 589\n",
      "Number of processed documents: 590\n",
      "Number of processed documents: 591\n",
      "Number of processed documents: 592\n",
      "Number of processed documents: 593\n",
      "Number of processed documents: 594\n",
      "Number of processed documents: 595\n",
      "Number of processed documents: 596\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 597\n",
      "Number of processed documents: 598\n",
      "Number of processed documents: 599\n",
      "Number of processed documents: 600\n",
      "Number of processed documents: 601\n",
      "Number of processed documents: 602\n",
      "Number of processed documents: 603\n",
      "Number of processed documents: 604\n",
      "Number of processed documents: 605\n",
      "Number of processed documents: 606\n",
      "Number of processed documents: 607\n",
      "Number of processed documents: 608\n",
      "Number of processed documents: 609\n",
      "Number of processed documents: 610\n",
      "Number of processed documents: 611\n",
      "Number of processed documents: 612\n",
      "Number of processed documents: 613\n",
      "Number of processed documents: 614\n",
      "Number of processed documents: 615\n",
      "Number of processed documents: 616\n",
      "Number of processed documents: 617\n",
      "Number of processed documents: 618\n",
      "Number of processed documents: 619\n",
      "Number of processed documents: 620\n",
      "Number of processed documents: 621\n",
      "Number of processed documents: 622\n",
      "Number of processed documents: 623\n",
      "Number of processed documents: 624\n",
      "Number of processed documents: 625\n",
      "Number of processed documents: 626\n",
      "Number of processed documents: 627\n",
      "Number of processed documents: 628\n",
      "Number of processed documents: 629\n",
      "Number of processed documents: 630\n",
      "Number of processed documents: 631\n",
      "Number of processed documents: 632\n",
      "Number of processed documents: 633\n",
      "Number of processed documents: 634\n",
      "Number of processed documents: 635\n",
      "Number of processed documents: 636\n",
      "Number of processed documents: 637\n",
      "Number of processed documents: 638\n",
      "Number of processed documents: 639\n",
      "Number of processed documents: 640\n",
      "Number of processed documents: 641\n",
      "Number of processed documents: 642\n",
      "Number of processed documents: 643\n",
      "Number of processed documents: 644\n",
      "Number of processed documents: 645\n",
      "Number of processed documents: 646\n",
      "Number of processed documents: 647\n",
      "Number of processed documents: 648\n",
      "Number of processed documents: 649\n",
      "Number of processed documents: 650\n",
      "Number of processed documents: 651\n",
      "Number of processed documents: 652\n",
      "Number of processed documents: 653\n",
      "Number of processed documents: 654\n",
      "Number of processed documents: 655\n",
      "Number of processed documents: 656\n",
      "Number of processed documents: 657\n",
      "Number of processed documents: 658\n",
      "Number of processed documents: 659\n",
      "Number of processed documents: 660\n",
      "Number of processed documents: 661\n",
      "Number of processed documents: 662\n",
      "Number of processed documents: 663\n",
      "Number of processed documents: 664\n",
      "Number of processed documents: 665\n",
      "Number of processed documents: 666\n",
      "Number of processed documents: 667\n",
      "Number of processed documents: 668\n",
      "Number of processed documents: 669\n",
      "Number of processed documents: 670\n",
      "Number of processed documents: 671\n",
      "Number of processed documents: 672\n",
      "Number of processed documents: 673\n",
      "Number of processed documents: 674\n",
      "Number of processed documents: 675\n",
      "Number of processed documents: 676\n",
      "Number of processed documents: 677\n",
      "Number of processed documents: 678\n",
      "Number of processed documents: 679\n",
      "Number of processed documents: 680\n",
      "Number of processed documents: 681\n",
      "Number of processed documents: 682\n",
      "Number of processed documents: 683\n",
      "Number of processed documents: 684\n",
      "Number of processed documents: 685\n",
      "Number of processed documents: 686\n",
      "Number of processed documents: 687\n",
      "Number of processed documents: 688\n",
      "Number of processed documents: 689\n",
      "Number of processed documents: 690\n",
      "Number of processed documents: 691\n",
      "Number of processed documents: 692\n",
      "Number of processed documents: 693\n",
      "Number of processed documents: 694\n",
      "Number of processed documents: 695\n",
      "Number of processed documents: 696\n",
      "Number of processed documents: 697\n",
      "Number of processed documents: 698\n",
      "Number of processed documents: 699\n",
      "Number of processed documents: 700\n",
      "Number of processed documents: 701\n",
      "Number of processed documents: 702\n",
      "Number of processed documents: 703\n",
      "Number of processed documents: 704\n",
      "Number of processed documents: 705\n",
      "Number of processed documents: 706\n",
      "Number of processed documents: 707\n",
      "Number of processed documents: 708\n",
      "Number of processed documents: 709\n",
      "Number of processed documents: 710\n",
      "Number of processed documents: 711\n",
      "Number of processed documents: 712\n",
      "Number of processed documents: 713\n",
      "Number of processed documents: 714\n",
      "Number of processed documents: 715\n",
      "Number of processed documents: 716\n",
      "Number of processed documents: 717\n",
      "Number of processed documents: 718\n",
      "Number of processed documents: 719\n",
      "Number of processed documents: 720\n",
      "Number of processed documents: 721\n",
      "Number of processed documents: 722\n",
      "Number of processed documents: 723\n",
      "Number of processed documents: 724\n",
      "Number of processed documents: 725\n",
      "Number of processed documents: 726\n",
      "Number of processed documents: 727\n",
      "Number of processed documents: 728\n",
      "Number of processed documents: 729\n",
      "Number of processed documents: 730\n",
      "Number of processed documents: 731\n",
      "Number of processed documents: 732\n",
      "Number of processed documents: 733\n",
      "Number of processed documents: 734\n",
      "Number of processed documents: 735\n",
      "Number of processed documents: 736\n",
      "Number of processed documents: 737\n",
      "Number of processed documents: 738\n",
      "Number of processed documents: 739\n",
      "Number of processed documents: 740\n",
      "Number of processed documents: 741\n",
      "Number of processed documents: 742\n",
      "Number of processed documents: 743\n",
      "Number of processed documents: 744\n",
      "Number of processed documents: 745\n",
      "Number of processed documents: 746\n",
      "Number of processed documents: 747\n",
      "Number of processed documents: 748\n",
      "Number of processed documents: 749\n",
      "Number of processed documents: 750\n",
      "Number of processed documents: 751\n",
      "Number of processed documents: 752\n",
      "Number of processed documents: 753\n",
      "Number of processed documents: 754\n",
      "Number of processed documents: 755\n",
      "Number of processed documents: 756\n",
      "Number of processed documents: 757\n",
      "Number of processed documents: 758\n",
      "Number of processed documents: 759\n",
      "Number of processed documents: 760\n",
      "Number of processed documents: 761\n",
      "Number of processed documents: 762\n",
      "Number of processed documents: 763\n",
      "Number of processed documents: 764\n",
      "Number of processed documents: 765\n",
      "Number of processed documents: 766\n",
      "Number of processed documents: 767\n",
      "Number of processed documents: 768\n",
      "Number of processed documents: 769\n",
      "Number of processed documents: 770\n",
      "Number of processed documents: 771\n",
      "Number of processed documents: 772\n",
      "Number of processed documents: 773\n",
      "Number of processed documents: 774\n",
      "Number of processed documents: 775\n",
      "Number of processed documents: 776\n",
      "Number of processed documents: 777\n",
      "Number of processed documents: 778\n",
      "Number of processed documents: 779\n",
      "Number of processed documents: 780\n",
      "Number of processed documents: 781\n",
      "Number of processed documents: 782\n",
      "Number of processed documents: 783\n",
      "Number of processed documents: 784\n",
      "Number of processed documents: 785\n",
      "Number of processed documents: 786\n",
      "Number of processed documents: 787\n",
      "Number of processed documents: 788\n",
      "Number of processed documents: 789\n",
      "Number of processed documents: 790\n",
      "Number of processed documents: 791\n",
      "Number of processed documents: 792\n",
      "Number of processed documents: 793\n",
      "Number of processed documents: 794\n",
      "Number of processed documents: 795\n",
      "Number of processed documents: 796\n",
      "Number of processed documents: 797\n",
      "Number of processed documents: 798\n",
      "Number of processed documents: 799\n",
      "Number of processed documents: 800\n",
      "Number of processed documents: 801\n",
      "Number of processed documents: 802\n",
      "Number of processed documents: 803\n",
      "Number of processed documents: 804\n",
      "Number of processed documents: 805\n",
      "Number of processed documents: 806\n",
      "Number of processed documents: 807\n",
      "Number of processed documents: 808\n",
      "Number of processed documents: 809\n",
      "Number of processed documents: 810\n",
      "Number of processed documents: 811\n",
      "Number of processed documents: 812\n",
      "Number of processed documents: 813\n",
      "Number of processed documents: 814\n",
      "Number of processed documents: 815\n",
      "Number of processed documents: 816\n",
      "Number of processed documents: 817\n",
      "Number of processed documents: 818\n",
      "Number of processed documents: 819\n",
      "Number of processed documents: 820\n",
      "Number of processed documents: 821\n",
      "Number of processed documents: 822\n",
      "Number of processed documents: 823\n",
      "Number of processed documents: 824\n",
      "Number of processed documents: 825\n",
      "Number of processed documents: 826\n",
      "Number of processed documents: 827\n",
      "Number of processed documents: 828\n",
      "Number of processed documents: 829\n",
      "Number of processed documents: 830\n",
      "Number of processed documents: 831\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of processed documents: 832\n",
      "Number of processed documents: 833\n",
      "Number of processed documents: 834\n",
      "Number of processed documents: 835\n",
      "Number of processed documents: 836\n",
      "Number of processed documents: 837\n",
      "Number of processed documents: 838\n",
      "Number of processed documents: 839\n",
      "Number of processed documents: 840\n",
      "Number of processed documents: 841\n",
      "Number of processed documents: 842\n",
      "Number of processed documents: 843\n",
      "Number of processed documents: 844\n",
      "Write to ../../data/sentiment/norec_sentence/test_pred.docbin...done\n"
     ]
    }
   ],
   "source": [
    "from norec_sentiment import FullSentimentAnnotator\n",
    "\n",
    "ann = FullSentimentAnnotator()\n",
    "ann.add_all()\n",
    "\n",
    "ann.annotate_docbin(\"../../data/sentiment/norec_sentence/train.docbin\",\n",
    "                    \"../../data/sentiment/norec_sentence/train_pred.docbin\")\n",
    "\n",
    "ann.annotate_docbin(\"../../data/sentiment/norec_sentence/dev.docbin\",\n",
    "                    \"../../data/sentiment/norec_sentence/dev_pred.docbin\")\n",
    "\n",
    "ann.annotate_docbin(\"../../data/sentiment/norec_sentence/test_pred.docbin\",\n",
    "                    \"../../data/sentiment/norec_sentence/test_pred.docbin\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can use the HMM model from skweak to aggregate all the predictions from the weak labelling functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "'IS_ALPHA'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-7-639785378ba9>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0munified_model\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mskweak\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0maggregation\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mHMM\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"hmm\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msequence_labelling\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0munified_model\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"../../data/sentiment/norec_sentence/train_pred.docbin\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m unified_model.annotate_docbin(\"../../data/sentiment/norec_sentence/train_pred.docbin\",\n\u001b[1;32m      4\u001b[0m                               \"../../data/sentiment/norec_sentence/train_pred.docbin\")\n\u001b[1;32m      5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Exps/skweak/skweak/aggregation.py\u001b[0m in \u001b[0;36mfit\u001b[0;34m(self, docbin_file, cutoff, n_iter, tol, cutoff_for_init)\u001b[0m\n\u001b[1;32m    330\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    331\u001b[0m         \u001b[0;31m# And add the counts from majority voter\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 332\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_add_mv_counts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdocs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    333\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    334\u001b[0m         \u001b[0;31m# Finally, we postprocess the counts and get probabilities\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Exps/skweak/skweak/aggregation.py\u001b[0m in \u001b[0;36m_add_mv_counts\u001b[0;34m(self, docs)\u001b[0m\n\u001b[1;32m    498\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    499\u001b[0m             \u001b[0;31m# We extract the observations\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 500\u001b[0;31m             \u001b[0mobs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_observation_df\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdoc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    501\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    502\u001b[0m             \u001b[0;31m# And aggregate the results\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Exps/skweak/skweak/aggregation.py\u001b[0m in \u001b[0;36mget_observation_df\u001b[0;34m(self, doc)\u001b[0m\n\u001b[1;32m    119\u001b[0m                 \u001b[0;32mfor\u001b[0m \u001b[0mspan\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdoc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mspans\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0msource\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    120\u001b[0m                     \u001b[0mspan_index\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0munique_spans_indices\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mspan\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstart\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mspan\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mend\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 121\u001b[0;31m                     \u001b[0mdata\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mspan_index\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msource_index\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlabel_indices\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mspan\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlabel_\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    122\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    123\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mpandas\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mDataFrame\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcolumns\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msources\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindex\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msorted_unique_spans\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyError\u001b[0m: 'IS_ALPHA'"
     ]
    }
   ],
   "source": [
    "unified_model = skweak.aggregation.HMM(\"hmm\", [0, 1, 2], sequence_labelling=False)\n",
    "unified_model.fit(\"../../data/sentiment/norec_sentence/train_pred.docbin\")\n",
    "unified_model.annotate_docbin(\"../../data/sentiment/norec_sentence/train_pred.docbin\",\n",
    "                              \"../../data/sentiment/norec_sentence/train_pred.docbin\")\n",
    "\n",
    "unified_model.annotate_docbin(\"../../data/sentiment/norec_sentence/dev_pred.docbin\",\n",
    "                              \"../../data/sentiment/norec_sentence/dev_pred.docbin\")\n",
    "\n",
    "unified_model.annotate_docbin(\"../../data/sentiment/norec_sentence/test_pred.docbin\",\n",
    "                              \"../../data/sentiment/norec_sentence/test_pred.docbin\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also compare with a majority voting aggregator, which will highlight the benefits of skweak."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mv = skweak.aggregation.MajorityVoter(\"mv\", [0, 1, 2], sequence_labelling=False) #type: ignore\n",
    "mv.annotate_docbin(\"../../data/sentiment/norec_sentence/test_pred.docbin\",\n",
    "                   \"../../data/sentiment/norec_sentence/test_pred.docbin\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can evaluate everything"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pred_docs = list(utils.docbin_reader(\"../../data/sentiment/norec_sentence/test_pred.docbin\"))\n",
    "gold = [d.user_data[\"gold\"] for d in pred_docs]\n",
    "\n",
    "# Evaluate the weak labelling approaches themselves\n",
    "for lexicon in pred_docs[0].user_data[\"spans\"].keys():\n",
    "    pred = []\n",
    "    for d in pred_docs:\n",
    "        for span in d.spans[lexicon]:\n",
    "            pred.append(span.label_)\n",
    "\n",
    "    lex_f1 = f1_score(gold, pred, average=\"macro\")\n",
    "    print(\"{0}:\\t{1:.3f}\".format(lexicon, lex_f1))\n",
    "    \n",
    "for aggregator in [\"mv\", \"hmm\"]:\n",
    "    pred = []\n",
    "    for d in pred_docs:\n",
    "        for span in d.spans[aggregator]:\n",
    "            pred.append(span.label_)\n",
    "    agg_f1 = f1_score(gold, pred, average=\"macro\")\n",
    "    print(\"{0}:\\t{1:.3f}\".format(aggregator, agg_f1))\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "myvenv",
   "language": "python",
   "name": "myvenv"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
